dstore-dist

Configuring dstore-dist to Receive and Forward Protobuf

dstore-dist is configured using the configuration file dstore-dist.yml, which by default is located in /etc/pdns-dstore-dist/.

To configure a basic dstore-dist instance that forwards protobuf to a 192.168.1.2 on port 1234, use the following configuration:

# Addresses to listen on for incoming connections.
listen:
  - ":2000"

# Destinations to write to.
destinations:
  # Arbitrary names that can be referred to in routes
  mydestination:
    addresses:
      - "192.168.1.2:1234"
# Routing and filtering
routes:
  myroute:
    destinations:
      - mydestination

Add a new destination with multiple IPs that dstore-dist will distribute traffic to in a round-robin fashion, as part of a new route that also filters out events that are not A records.

# Addresses to listen on for incoming connections.
listen:
  - ":2000"

# Destinations to write to.
destinations:
  # Arbitrary names that can be referred to in routes
  mydestination:
    addresses:
      - "192.168.1.2:1234"
  newdestination:
    addresses:
      - "10.43.2.1:5678"
      - "10.43.2.2:5678"
    distribute: roundrobin
# Routing and filtering
routes:
  myroute:
    destinations:
      - mydestination
  newroute:
    destinations:
      - newdestination
    filters:
      # All of these checks have to pass
      - qtype: A

The following configuration shows dstore-dist forwarding to Kafka. There are two Kafka destinations configured; kafkaproto receives protobuf messages, while the kafkajson destination receives messages that have first been converted to JSON, and also compressed using gzip compression.

# Addresses to listen on for incoming connections.
listen:
  - ":2000"

# Destinations to write to.
destinations:
  # Arbitrary names that can be referred to in routes
  kafkaproto:
    type: kafka
    kafka:
      # At least one address needs to be specified
      addresses:
        - "kafka1.example.com:29092"
      # topic is mandatory
      topic: dstore-dist
  kafkajson:
    type: kafka
    kafka:
      # At least one address needs to be specified
      addresses:
        - "kafka2.example.com:29092"
      # topic is mandatory
      topic: dstore-dist
      json_encode: true
      compression: gzip
routes:
  myroute:
    destinations:
      - kafkaproto
      - kafkajson

For more information on the configuration of dstore-dist, see dstore-dist Configuration.