ClickHouse

Description

This ClickHouse destination can be used to send metrics, logs, and traces to a ClickHouse server.

Supported Types

Metrics	Logs	Traces	Bindplane Collector
✓	✓	✓	v1.41.0+

Configuration

Field	Description
Telemetry Types	The kind of telemetry that will be sent to the ClickHouse server. Can be any combination of logs, metrics, and traces.
Protocol	The kind of protocol to be used when sending to the ClickHouse server. Can be TCP, HTTP, or HTTPS. See this ClickHouse documentation for more information.
Endpoint	The endpoint to use to send telemetry data to ClickHouse. Supports multiple endpoints. See this ClickHouse documentation for more information.
Username	Username to use to authenticate with the ClickHouse server. See this ClickHouse documentation for more information.
Password	Password to use to authenticate with the ClickHouse server. See this ClickHouse documentation for more information.
Database	Name of the database to use when interacting with the ClickHouse server.
Logs Table Name	Name of the table inside Database in ClickHouse to store log data in. Creates the table if it does not already exist.
Metrics Table Name	Name of the table inside Database in ClickHouse to store metric data in. Creates the table if it does not already exist.
Traces Table Name	Name of the table inside Database in ClickHouse to store traces data in. Creates the table if it does not already exist.
TTL	The data time-to-live, for example '30m'. 0 means no TTL. Make sure the telemetry sent has a timestamp field.
Timeout	Timeout for each attempt to send data to ClickHouse.
Connection Parameters	Additional connection parameters with map format. Used as query parameters in the URL. See this ClickHouse documentation for more information.

IMPORTANT

Metrics stored in ClickHouse are grouped by their type (sum, gauge, etc.) and stored in tables specific to that type by appending _type to the table name. For example, using the example configuration below, gauge metrics would be stored in bpop.bp_metrics_gauge. To read more, see this documentation.

NOTE

TLS can be configured using Connection Parameters. To do so, add an entry called `secure` with the value true. An example can be seen below.

WARNING

The collector will fail to start if it cannot reach the ClickHouse server on start up. If the ClickHouse server becomes unreachable while the collector is already running, it will continue running while reporting an error.

The collector can also fail to start if configured to use TTL, but the telemetry being sent lacks a Timestamp field.

This destination supports the following retry and queuing settings:

Sending Queue	Persistent Queue	Retry on Failure
✓	✓	✓

Example Configuration

Here we set a single endpoint that uses the TCP protocol. We are sending metrics, logs, and traces, so we have them selected and table names for them. We also utilize authentication and are setting compression and TLS via the connection parameters. Finally, we have the sending queue, persistent queue, and retry on failure enabled as well.

Web Interface

Standalone Destination

apiVersion: bindplane.observiq.com/v1
kind: Destination
metadata:
  id: clickhouse
  name: clickhouse
spec:
  type: clickhouse
  parameters:
    - name: telemetry_types
      value: ['Logs', 'Metrics', 'Traces']
    - name: protocol
      value: tcp
    - name: endpoint
      value: ['0.0.0.0:9000']
    - name: username
      value: admin
    - name: password
      value: admin
    - name: database
      value: bpop
    - name: logs_table_name
      value: bp_logs
    - name: metrics_table_name
      value: bp_metrics
    - name: traces_table_name
      value: bp_traces
    - name: connection_params
      value:
        compress: gzip
        secure: true
    - name: retry_on_failure_enabled
      value: 'true'
    - name: sending_queue_enabled
      value: 'true'
    - name: persistent_queue_enabled
      value: 'true'