Overview¶
How It Works¶
PowerDNS dstore consists of software for the processing and distribution of protobuf log messages from
PowerDNS Recursor, DNSdist and Authoritative server.
All the software components in dstore accept events over the network and process them in some way;
for example distributing them to different destinations, converting them to different formats, aggregating them into reports etc.
Each event contains a DNS message (source address, query name, query type, etc.) representing either a DNS query or a DNS
query+response augmented by performance counters. In this documentation, the terms event and message are used interchangeably.
dstore receives events wrapped in Protocol Buffers format (a.k.a. protobuf).
These are a language-neutral, platform-neutral, extensible mechanism for serializing structured data.
The events destined for dstore can be emitted by a variety of sources:
- PowerDNS Recursor
- PowerDNS DNSdist
- PowerDNS Authoritative Server (not currently, but potentially in a future version)
dnspcap2protobuffrom the PowerDNS distribution
For information on how to configure DNSdist and Recursor to emit protobuf, and example configuration for the dstore components, see Getting Started.
Components¶
dstore consists of a number of components which interact with each other, as shown in the figure below.
dstore-dist¶
dstore-dist is the primary daemon which listens for events on TCP port 2000, and performs a variety of processing
on those events, including filtering, distribution to different destinations, and conversion to new protocols such as Kafka.
It is responsible for listening for events, decoding protobufs, and the onwards forwarding of events.
dstore-dist-top-reporter¶
dstore-dist-top-reporter is used to generate reports from protobuf messages. These reports are usually
based on sampled data (dstore-dist can be used to perform the data sampling before passing the events to dstore-dist-top-reporter).
Reports are stored in an Elastic/Opensearch backend.
dstore-dist-eventforwarder¶
dstore-dist-eventforwarder is used to store reports about events related to DNS filtering in Elastic/Opensearch. Those reports
can then later be retrieved/searched by the report-api component.
dnspcap2protobuf¶
dnspcap2protobuf utility from the PowerDNS distribution reads a pcap file and converts it to protobuf format which you can subsequently feed to dstore-dist:
The following code excerpt will capture some DNS traffic, and then send it to dstore-dist in protobuf format on port 2000.