dstore-dist-eventforwarder

Command Arguments¶

dstore-dist-eventforwarder [-config file] [-debug] [-addr address:port] [-nconcur num] [-es-addr url] [-es-user user] [-es-pass password] [-es-cloud-id cloud-id] [-es-api-key api-key] [-es-index index-name]

Description¶

dstore-dist-eventforwarder receives events from dstore-dist, converts them to a JSON format that is suitable for ingestion by external services such as elasticsearch, and forwards the events on to those services. The initial version only supports forwarding the events to Elastic/Opensearch, although future versions may support forwarding to other services, such as a generic HTTP Webhook endpoint. dstore-dist-eventforwarder differs from dstore-dist in that it both simplifies the event data (removing many of the low-level fields, and any response data), while simultaneously enriching it by attempting to categorize events based on the tags present in those events. The dstore-dist-eventforwarder is only suitable for forwarding events filtered by PowerDNS Platform Filter, which can be achieved by for example using the has_tags and is_response filters in dstore-dist.

Flags¶

Flag	Argument	Description
-config	`<file>`	Load configuration from `<file>`
-debug		Generate debug logging
-addr	`<ip:port>`	Address/Port to listen on for blocked messages
-es-api-key	`<api-key>`	Elastic/Opensearch API key
-es-cloud-id	`<cloud-id>`	Elastic/Opensearch cloud id
-es-index	`<index-name>`	Elastic/Opensearch index name
-es-user	`<username>`	Elastic/Opensearch username
-es-pass	`<password>`	Elastic/Opensearch password
-es-addr	`<url>`	Elastic/Opensearch URL to connect to
-help		Display a helpful message and exit
-nconcur	`<num>`	Number of concurrent senders, a value <= 0 means default (number of CPUs)
-qaddr	`<ip:port>`	Address/Port to listen on for query count messages
-qnconcur	`<num>`	Number of concurrent senders, a value <= 0 means default (number of CPUs)

Files¶

/etc/pdns-dstore-dist/dstore-dist-eventforwarder.yml: Default location of the config file

Configuration¶

See Configuration of dstore-dist-eventforwarder for details of the configuration file format.

Type and Format of Events¶

The event forwarder creates three types of event for storage:

dnsfilter events - An event is created for every per-subscriber filtering event, with details of the event such as the domain name that was blocked, and the type of filter (malware, contentfilter etc.). The source for these events is all messages received via the listen address specified with -addr or the configuration key listen:.
dnsglobalfilter events - These events are created periodically and are using to count filtering stats on a global basis. The source is the same as for dnsfilter events.
dnsquery events - These events are created periodically and are based on messages received via the listen address specified with -qaddr or the configuration key query_count.listen_addr. The dnsquery events count all queries sent by subscribers (and also global query counts). These events support sampling by the upstream dstore-dist, i.e. the counts are multiplied by the sampling rate specified by the query_count.sample_rate

dnsfilter Event JSON¶

The following JSON fields can be present in an event of type dnsfilter:

Field Name	Type	Optional?	Description
type	string	no	Always has a value of `dnsfilter`
app	string	no	Always has a value of `pdns`
user_id	string	no	Contains the name of the subscriber that the filtering event relates to
profile_name	string	yes	The name of the subscriber profile that triggered the event
qname	string	yes	The domain name of the query
device_id	string	yes	The ID of the device that triggered the event
device_ip	string	yes	The IP address that sent the DNS query
filter_type	string	yes	One of `contentfilter`, `malware`, `botnet`, `phishing`, `blocklist`
rule	string	yes	The filtering rule that triggered the event
tags	array of strings	yes	All the tags present in the original message
categories	array of strings	yes	The categories associated with the event, e.g. "adult", or "gambling"
timestamp	integer	yes	The timestamp in milliseconds since UNIX epoch
timestamp_iso	string	yes	ISO 8601 format timestamp string

dnsglobalfilter Event JSON¶

The following JSON fields can be present in an event of type dnsglobalfilter:

Field Name	Type	Optional?	Description
type	string	no	Always has a value of `dnsglobalfilter`
app	string	no	Always has a value of `pdns`
user_id	string	no	Always the empty string ""
blocked_count	string	yes	Contains a count of the total number of blocking events. Mutually exclusive with the other optional fields.
category	string	yes	Contains a category name. Always accompanied by `category_count` field. Otherwise, mutually exclusive with the other optional fields.
category_count	integer	yes	Contains a count for the category. Always accompanied by `category`. Otherwise, mutually exclusive with the other optional fields.
filter_type	string	yes	Same values as `filter_type` in `dnsfilter`. Always accompanied by `filter_type_count`. Otherwise, mutually exclusive with the other optional fields.
filter_type_count	integer	yes	Contains a count of filter_type. Always accompanied by `filter_type`. Otherwise, mutually exclusive with the other optional fields.
timestamp	integer	yes	The timestamp in milliseconds since UNIX epoch
timestamp_iso	string	yes	ISO 8601 format timestamp string

dnsquery JSON¶

The following JSON fields can be present in an event of type dnsquery:

Field Name	Type	Optional?	Description
type	string	no	Always has a value of `dnsquery`
app	string	no	Always has a value of `pdns`
user_id	string	no	Contains the name of the subscriber that the query counts refer to. For global counts, always the empty string ""
query_count	integer	yes	Contains a count of the total number of events.
timestamp	integer	yes	The timestamp in milliseconds since UNIX epoch
timestamp_iso	string	yes	ISO 8601 format timestamp string