Configuration Reference

Monitoring

The monitoring Helm Chart can be configured via the following parameters:

Parameter	Type	Description
`global`	Global	Configuration options for important global usage within the Cloud Control Helm Charts
`grafana`	Grafana	Configuration options for Grafana
`kube-state-metrics`	Kube-state-metrics	Configuration options for Kube-state-metrics
`prometheus`	Prometheus	Configuration options for Prometheus
`prometheus-adapter`	PrometheusAdapter	Configuration options for Prometheus Adapter

Grafana

You can configure Grafana using the following parameters:

Parameter	Type	Default	Description
`affinity`	`k8s:`Affinity	`{}`	pod affinity (Kubernetes docs: Affinity and anti-affinity)
`config`	`ext:`GrafanaConfig	`log: mode: "console" level: "warn" auth: disable_login_form: False disable_signout_menu: False auth.anonymous: enabled: False`	Grafana configuration Note: This allows for a large amount of flexibility (to configure an additional authentication provider for example), but can also break Grafana if misconfigured.
`containerSecurityContext`	`k8s:`SecurityContext	`allowPrivilegeEscalation: false readOnlyRootFilesystem: true seccompProfile: type: RuntimeDefault capabilities: drop: - "ALL"`	SecurityContext applied to each container
`dashboards`	`boolean`	`true`	If true, deploy the Grafana dashboards for PowerDNS products
`dashboardLabels`	`dictionary`	`{}`	Set of `key:value` pairs to be as labels on each dashboard. Example: `dashboardLabels: my.label/name: some-value`
`enabled`	`boolean`	`true`	If true, deploy Grafana
`ingress`	MonitoringIngress	`{}`	Ingress configuration
`nodeSelector`	`k8s:`NodeSelector	`{}`	Kubernetes pod nodeSelector
`podAnnotations`	`k8s:`Annotations	`{}`	Annotations to be added to each pod
`podSecurityContext`	`k8s:`PodSecurityContext	`fsGroup: 2000 runAsUser: 1000 runAsGroup: 2000 runAsNonRoot: true`	SecurityContext applied to each pod
`resources`	`k8s:`Resources	`{}`	Resources allocated to the grafana container
`service`	GrafanaService		Service object for Grafana. Default: A `ClusterIP` service listening on port 3000
`tolerations`	List of `k8s:`Tolerations	`[]`	Kubernetes pod Tolerations

Note: If you have Grafana running already in your cluster via the Grafana Operator, you can set enabled: false and leave the default dashboards: true to only deploy the dashboards.

Grafana Service

Parameters to configure the service object for Grafana. For example:

grafana:
  service:
    type: LoadBalancer
    annotations:
      metallb.universe.tf/address-pool: name_of_pool

Parameter	Type	Default	Description
`annotations`	`k8s:`Annotations	`{}`	Annotations for the service
`clusterIP`	`string`		Static cluster IP, must be in the cluster's range of cluster IPs and not in use. Randomly assigned when not specified.
`loadBalancerClass`	`string`		Used to select a non-default type of LoadBalancer class to ensure the appropriate LoadBalancer provisioner attempt to manage this LoadBalancer service
`type`	`string`	`ClusterIP`	Type of service. Available options: `"ClusterIP"` `"LoadBalancer"` `"NodePort"`

Kube-state-metrics

You can configure Kube-state-metrics using the following parameters:

Parameter	Type	Default	Description
`affinity`	`k8s:`Affinity	`{}`	pod affinity (Kubernetes docs: Affinity and anti-affinity)
`containerSecurityContext`	`k8s:`SecurityContext	`allowPrivilegeEscalation: false readOnlyRootFilesystem: true seccompProfile: type: RuntimeDefault capabilities: drop: - "ALL"`	SecurityContext applied to each container
`enabled`	`boolean`	`true`	If true, deploy Kube-state-metrics
`nodeSelector`	`k8s:`NodeSelector	`{}`	Kubernetes pod nodeSelector
`podAnnotations`	`k8s:`Annotations	`{}`	Annotations to be added to each pod
`podSecurityContext`	`k8s:`PodSecurityContext	`fsGroup: 65534 runAsUser: 65534 runAsGroup: 65534 runAsNonRoot: true`	SecurityContext applied to each pod
`replicas`	`integer`	`1`	Number of replicas
`resources`	`k8s:`Resources	`{}`	Resources allocated to the grafana container
`tolerations`	List of `k8s:`Tolerations	`[]`	Kubernetes pod Tolerations

Prometheus

You can configure Prometheus using the following parameters:

Parameter	Type	Default	Description
`affinity`	`k8s:`Affinity	`{}`	pod affinity (Kubernetes docs: Affinity and anti-affinity)
`alertRules`	`boolean`	`true`	If true, deploy the Prometheus alert rules for PowerDNS products
`containerSecurityContext`	`k8s:`SecurityContext	`allowPrivilegeEscalation: false readOnlyRootFilesystem: true seccompProfile: type: RuntimeDefault capabilities: drop: - "ALL"`	SecurityContext applied to each container
`enabled`	`boolean`	`true`	If true, deploy Prometheus
`externalLabels`	`dictionary`	`{}`	Set of `key:value` pairs to be configured on Prometheus to be used when metrics are exposed to a larger mesh of Prometheus instances (for example when using Thanos). Example: `externalLabels: location: eu-central-1`
`ingress`	MonitoringIngress	`{}`	Ingress configuration
`nodeSelector`	`k8s:`NodeSelector	`{}`	Kubernetes pod nodeSelector
`podAnnotations`	`k8s:`Annotations	`{}`	Annotations to be added to each pod
`podSecurityContext`	`k8s:`PodSecurityContext	`fsGroup: 2000 runAsUser: 1000 runAsGroup: 2000 runAsNonRoot: true`	SecurityContext applied to each pod
`replicas`	`integer`	`1`	Number of replicas
`resources`	`k8s:`Resources	`{}`	Resources allocated to the prometheus container
`service`	PrometheusService		Service object for Prometheus. Default: A `ClusterIP` service listening on port 9090
`thanos`	ThanosConfiguration		Configuration of Thanos
`tolerations`	List of `k8s:`Tolerations	`[]`	Kubernetes pod Tolerations

Prometheus Service

Parameters to configure the service object for Prometheus. For example:

prometheus:
  service:
    type: LoadBalancer
    annotations:
      metallb.universe.tf/address-pool: name_of_pool

Parameter	Type	Default	Description
`allocateLoadBalancerNodePorts`	`boolean`	`true`	If true, services with type `LoadBalancer` automatically assign NodePorts. Can be set to `false` if the LoadBalancer provider does not rely on NodePorts
`annotations`	`k8s:`Annotations	`{}`	Annotations for the service
`clusterIP`	`string`		Static cluster IP, must be in the cluster's range of cluster IPs and not in use. Randomly assigned when not specified.
`clusterIPs`	List of `string`		List of static cluster IPs, must be in the cluster's range of cluster IPs and not in use.
`externalIPs`	List of `string`		List of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes and must be user-defined on the cluster's nodes
`externalTrafficPolicy`	`string`	`Cluster`	Can be set to `Local` to let nodes distribute traffic received on one of the externally-facing addresses (`NodePort` and `LoadBalancer`) solely to endpoints on the node itself
`healthCheckNodePort`	`integer`		For services with type `LoadBalancer` and externalTrafficPolicy `Local` you can configure this value to choose a static port for the NodePort which external systems (LoadBalancer provider mainly) can use to determine which node holds endpoints for this service
`internalTrafficPolicy`	`string`	`Cluster`	Can be set to `Local` to let nodes distribute traffic received on the `ClusterIP` solely to endpoints on the node itself
`loadBalancerIP`	`string`		Deprecated Kubernetes feature, available for backwards compatibility: IP address to attempt to claim for use by this LoadBalancer. Replaced by `annotations` specific to each LoadBalancer provider
`loadBalancerSourceRanges`	List of `string`		If supported by the LoadBalancer provider, restrict traffic to this LoadBalancer to these ranges
`loadBalancerClass`	`string`		Used to select a non-default type of LoadBalancer class to ensure the appropriate LoadBalancer provisioner attempt to manage this LoadBalancer service
`publishNotReadyAddresses`	`boolean`	`false`	Service is populated with endpoints regardless of readiness state
`sessionAffinity`	`string`	`None`	Can be set to `ClientIP` to attempt to maintain session affinity.
`sessionAffinityConfig`	`k8s:`SessionAffinityConfig	`{}`	Configuration of session affinity
`type`	`string`	`ClusterIP`	Type of service. Available options: `"ClusterIP"` `"LoadBalancer"` `"NodePort"`

Thanos Configuration

You can use the enabled parameter inside thanos to enable Thanos. For example:

prometheus:
  externalLabels:
    datacenter: dc1
  thanos:
    enabled: true

Note: externalLabels is recommended to be configured when Thanos is enabled to ensure you can distinguish between metrics gathered from the different locations.

When Thanos is enabled, the following happens:

Prometheus pods have an extra container: thanos-sidecar
A GRPC ClusterIP Service is created named prometheus-thanos-grpc
An HTTP ClusterIP Service is created named prometheus-thanos-http

To further configure these endpoints, you can refer to the following:

Thanos services
Thanos GRPC TLS
Thanos HTTP ingress

Thanos services

To configure the Thanos service objects, you can use the same syntax as the generic PrometheusService. For example:

prometheus:
  externalLabels:
    datacenter: dc1
  thanos:
    enabled: true

    grpc:
      service:
        type: LoadBalancer
        annotations:
          metallb.universe.tf/address-pool: name_of_pool

    http:
      service:
        type: LoadBalancer
        annotations:
          metallb.universe.tf/address-pool: name_of_pool

Thanos GRPC TLS

For the GRPC communication, you can configure TLS. The tls parameter allows selection of an existing TLS secret with a tls.key & tls.crt to use for enabling TLS on incoming traffic. The clientca parameter needs a secret to be available with a ca.crt data item which it can use to validate outgoing connections to TLS enabled endpoints.

For example:

prometheus:
  externalLabels:
    datacenter: dc1
  thanos:
    enabled: true

    grpc:
      service:
        type: LoadBalancer
        annotations:
          metallb.universe.tf/address-pool: name_of_pool
      tls:
        secret: my-tls-secret
      clientca:
        secret: my-ca-secret

    http:
      service:
        type: LoadBalancer
        annotations:
          metallb.universe.tf/address-pool: name_of_pool

Thanos HTTP ingress

For the inbound HTTP communication, you can configure an ingress. The ingress parameter takes the same syntax as the generic MonitoringIngress.

For example:

prometheus:
  externalLabels:
    datacenter: dc1
  thanos:
    enabled: true

    http:
      ingress:
        enabled: true
        ingressClassName: "nginx"
        hosts:
          - thanos.example.local
        tls:
          - secretName: thanos-ingress-cert
            hosts:
            - thanos.example.local

Prometheus Adapter

You can configure Prometheus Adapter using the following parameters:

Parameter	Type	Default	Description
`affinity`	`k8s:`Affinity	`{}`	pod affinity (Kubernetes docs: Affinity and anti-affinity)
`containerSecurityContext`	`k8s:`SecurityContext	`allowPrivilegeEscalation: false readOnlyRootFilesystem: true seccompProfile: type: RuntimeDefault capabilities: drop: - "ALL"`	SecurityContext applied to each container
`enabled`	`boolean`	`true`	If true, deploy Kube-state-metrics
`nodeSelector`	`k8s:`NodeSelector	`{}`	Kubernetes pod nodeSelector
`podAnnotations`	`k8s:`Annotations	`{}`	Annotations to be added to each pod
`podSecurityContext`	`k8s:`PodSecurityContext	`fsGroup: 10001 runAsUser: 10001 runAsGroup: 10001 runAsNonRoot: true`	SecurityContext applied to each pod
`replicas`	`integer`	`1`	Number of replicas
`resources`	`k8s:`Resources	`{}`	Resources allocated to the grafana container
`rules`	PrometheusAdapterRules		Configuration of rules for Prometheus Adapter, see PrometheusAdapterRules for more information
`tolerations`	List of `k8s:`Tolerations	`[]`	Kubernetes pod Tolerations

Prometheus Adapter Rules

By default, Prometheus Adapter in Cloud Control is configured with a few basic rules for calculating metrics. An example of one such rule:

- seriesQuery: 'dnsdist_queries'
  resources:
    overrides:
      namespace: {resource: "namespace"}
      pod: {resource: "pod"}
  name:
    matches: "^(.*)"
    as: "${1}_per_second"
  metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[1m])) by (<<.GroupBy>>)'

In the above rule, a metric is exposed to the Kubernetes API named dnsdist_queries_per_second, which is based on the rate at which the dnsdist_queries value increases per second. This metric will be available for each pod which has a corresponding value for dnsdist_queries in Prometheus. For more information on this syntax, refer to: Prometheus Adapter documentation.

To supply your own rules, you can use the following parameter:

prometheus-adapter:
  rules:
    custom:
      - seriesQuery: 'dnsdist_queries'
        resources:
          overrides:
            namespace: {resource: "namespace"}
            pod: {resource: "pod"}
        name:
          matches: "^(.*)"
          as: "${1}_per_second"
        metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[1m])) by (<<.GroupBy>>)'

Monitoring Ingress

Parameters to configure the ingress object for several components in a Monitoring deployment. For example for Grafana:

grafana:
  ingress:
    enabled: true
    ingressClassName: "nginx"

Parameter	Type	Default	Description
`annotations`	`k8s:`Annotations	`{}`	Annotations for the ingress
`enabled`	`boolean`	`false`	If true, create an ingress object
`hosts`	List of `string`	`[]`	List of hosts for which the ingress should accept traffic (If using `tls`, this should also include the `hosts` listed on the `tls` object)
`ingressClassName`	`string`		Name of the ingress class
`tls`	List of IngressTLS	`[]`	List of ingress tls configurations

Ingress TLS

Parameters to configure TLS for an ingress. For example for Grafana:

grafana:
  ingress:
    enabled: true
    ingressClassName: "nginx"
    hosts:
      - grafana.example.local
    tls:
      - secretName: grafana-ingress-cert
        hosts:
        - grafana.example.local

The above example will result in an ingress for Grafana listening for grafana.example.local and serving the certificate contained in the secret grafana-ingress-cert.

Note: tls is a list of configurations. Below shows the parameters which can be used for each entry in that list:

Parameter	Type	Default	Description
`hosts`	List of `string`	`[]`	List of hosts for which the ingress should accept TLS traffic
`secretName`	`string`	`""`	Name of a secret containing the certificate to be used for this ingress

Monitoring Operators

The monitoring-operators Helm Chart can be configured via the following parameters:

Parameter	Type	Description
`global`	Global	Configuration options for important global usage within the Cloud Control Helm Charts
`grafana-operator`	GrafanaOperator	Configuration options for Grafana Operator
`prometheus-operator`	PrometheusOperator	Configuration options for Prometheus Operator

Grafana Operator

You can configure Grafana Operator using the following parameters:

Parameter	Type	Default	Description
`affinity`	`k8s:`Affinity		pod affinity (Kubernetes docs: Affinity and anti-affinity). If unset, a default anti-affinity is applied using `antiAffinityPreset` to spread pods across nodes
`antiAffinityPreset`	`string`	`"preferred"`	pod anti affinity preset. Available options: `"preferred"` `"required"`
`containerSecurityContext`	`k8s:`SecurityContext	`allowPrivilegeEscalation: false readOnlyRootFilesystem: true seccompProfile: type: RuntimeDefault capabilities: drop: - "ALL"`	SecurityContext applied to each container
`enabled`	`boolean`	`true`	If true, deploy Grafana Operator
`nodeSelector`	`k8s:`NodeSelector	`{}`	Kubernetes pod nodeSelector
`podAnnotations`	`k8s:`Annotations	`{}`	Annotations to be added to each pod
`podSecurityContext`	`k8s:`PodSecurityContext	`fsGroup: 2000 runAsUser: 1000 runAsGroup: 2000 runAsNonRoot: true`	SecurityContext applied to each pod
`tolerations`	List of `k8s:`Tolerations	`[]`	Kubernetes pod Tolerations

Prometheus Operator

You can configure Prometheus Operator using the following parameters:

Parameter	Type	Default	Description
`affinity`	`k8s:`Affinity		pod affinity (Kubernetes docs: Affinity and anti-affinity). If unset, a default anti-affinity is applied using `antiAffinityPreset` to spread pods across nodes
`antiAffinityPreset`	`string`	`"preferred"`	pod anti affinity preset. Available options: `"preferred"` `"required"`
`containerSecurityContext`	`k8s:`SecurityContext	`allowPrivilegeEscalation: false readOnlyRootFilesystem: true seccompProfile: type: RuntimeDefault capabilities: drop: - "ALL"`	SecurityContext applied to each container
`enabled`	`boolean`	`true`	If true, deploy Prometheus Operator
`nodeSelector`	`k8s:`NodeSelector	`{}`	Kubernetes pod nodeSelector
`podAnnotations`	`k8s:`Annotations	`{}`	Annotations to be added to each pod
`podSecurityContext`	`k8s:`PodSecurityContext	`fsGroup: 65534 runAsUser: 65534 runAsGroup: 65534 runAsNonRoot: true`	SecurityContext applied to each pod
`tolerations`	List of `k8s:`Tolerations	`[]`	Kubernetes pod Tolerations

Global

Global options for these Helm Charts allow for the configuration of:

Image pull secrets to configure access to the OX registry or a private cache/intermediary
Compatibility mode for supported non-standard Kubernetes platforms

Example of using global to configure a private registry where you stored copies of the Cloud Control container images:

global:
  image:
    registry: "myregistry.local:8085"
    repository: "myrepository"

Or to configure Cloud Control to use a pre-existing Secret containing your registry credentials named my-registry-credentials:

global:
  imagePullSecretsList:
    - my-registry-credentials

Parameters which can be used:

Parameter	Type	Default	Description
`image`	ImageOverrides	`{}`	Overrides to configure where container images are pulled from. Default: The OX registry
`imagePullSecrets`	Map of ImagePullSecret	`{}`	Image pull secrets for which Secrets should be created and then used by the service accounts to pull container images from the registry. Recommendation: pre-provision the actual Secrets in your namespace and reference them using `imagePullSecretsList`
`imagePullSecretsList`	List of `string`	`[]`	List of names of Secrets which should be used by service accounts to pull container images from the registry
`openshift`	OpenShift	`{}`	Configuration of OpenSHift compatibility mode. Default: disabled

Image Overrides

You can configure the Helm Chart to ensure Kubernetes pulls container images from a different location than the OX registry. For example:

global:
  image:
    registry: "myregistry.local:8085"
    repository: "myrepository"

Parameters which can be used:

Parameter	Type	Default	Description
`registry`	`string`	`registry.open-xchange.com`	Override the base hostname of the URI from where container images are pulled
`repository`	`string`		Override the repository from which the container images are pulled. Default: Varies based on the type of component
`pullPolicy`	`string`	`"IfNotPresent"`	Force an image pull policy for all containers

Image Pull Secret

You can configure the Helm Chart to create Secrets for one or more sets of credentials to use to authenticate against a registry. Each entry should be a key-value pair, with:

key: Name of the secret
value: Dictionary holding the configuration of the secret

For example, to have an image pull secret with the name myIPSSecret to authenticate against the OX registry:

global:
  image:
    imagePullSecrets:
      myIPSSecret:
        registry: registry.open-xchange.com
        username: <USERNAME>
        password: <PASSWORD>
        email: admin@example.com

Parameters which can be used:

Parameter	Type	Required	Description
`registry`	`string`	`yes`	Base URI of the registry
`username`	`string`	`yes`	Username to authenticate with
`password`	`string`	`yes`	Password for authentication
`email`	`string`	`yes`	Email address to satisfy Kubernetes requirements for an image pull secrets in the desired format. Can contain dummy data as long as it satisfies the standard format of an email address

OpenShift

OpenShift requires some specific default settings in Cloud Control to be adjusted to satisfy the platform's requirements. You can configure this Helm Chart to deploy in OpenShift compatibility mode using the following example:

global:
  openshift:
    enabled: true

Parameters which can be used:

Parameter	Type	Default	Description
`openshift`	`boolean`	`false`	If true, enable OpenShift compatibility mode