Skip to content

dstore-dist-top-reporter

Configuring dstore-dist-top-reporter to Send Reports

dstore-dist is configured using the configuration file dstore-dist-top-reporter.yml, which by default is located in /etc/pdns-dstore-dist/.

Typically, data is sampled before sending to dstore-dist-top-reporter; this can be achieved in dstore-dist configuration using the sample field for a destination. For example to send only one message in one thousand: 

destinations:
  # Arbitrary names that can be referred to in routes
  mydestination:
    addresses:
      - "192.168.1.2:1234"
    sample: 1000

An example configuration file for dstore-dist-top-reporter is shown below: 

# Streams are different ports that dstore-dist will write filtered, sampled queries to.
# More than one report can subscribe to a single stream.
streams:
  - name: all-queries
    title: "All traffic (sampled)"
    address: ":4801"
    # This needs to match the sample value configured in dstore-dist
    upstream_sampling: 1000 

# Reports are generated from streams.
reports:
- name: all-tldplusone-domains
  field: qname/suffix+1
  # We always want to oversample, otherwise the summary data will be skewed
  n: 5000
  stream: all-queries
  interval: 60s
- name: all-domains
  field: qname
  n: 10000
  stream: all-queries
  interval: 60s

storage:
  - name: elasticsearch
    backend: elastic
    skip_empty: true
    url: http://elasticsearch:9200/
    elastic_index_template: "{{.ReportName}}-{{.TimestampDate}}"

The above configuration will use Elasticsearch for storage of the reports, and will generate two types of report:

  • all-tldplusone-domains: This report uses the qname/suffix+1 field as the key, e.g. example.com, google.co.uk. Stats for subdomains will be incorporated into the parent domain. This is the most useful type of "per-domain" report, as it aggregates domains at the level most likely to be interesting at a system-wide level.
  • all-domains: This report uses the lowercase query name as the key, so for example maps.google.co.uk or api.microsoft.com. It is useful for seeing the exact query domains made by clients.

For more information on the configuration of dstore-dist-top-reporter, see dstore-dist-top-reporter Configuration.