Kafka

Supported Types

Logs	Metrics	Traces
✓	✓	✓

Configuration Table

Parameter	Type	Default	Description
telemetry_types	telemetrySelector	["Logs", "Metrics", "Traces"]	Specifies which types of telemetry to export.
protocol_version	enum	"2.0.0"	The Kafka protocol version to use when communicating with brokers. Valid values are: "2.2.1", "2.2.0", "2.0.0", or "1.0.0".
brokers	strings	localhost:9092	A list of the brokers to connect to when sending metrics, traces, and logs.
timeout	int	5	Timeout (seconds) for every attempt to send data to the backend.
log_topic	string	otlp_logs	The name of the topic to export logs to.
log_encoding	enum	otlp_proto	The encoding to use when publishing logs to Kafka. Options are otlp_proto, otlp_json, and raw.
metric_topic	string	otlp_metrics	The name of the topic to export metrics to.
metric_encoding	enum	otlp_proto	The encoding to use when publishing metrics to Kafka. Options are otlp_proto and otlp_json.
trace_topic	string	otlp_spans	The name of the topic to export traces to.
trace_encoding	enum	oltp_proto	The encoding to use when publishing traces to Kafka. Options are otlp_proto, otlp_json, jaeger_proto, jeager_json, zipkin_proto, and zipkin_json.
compression	enum	gzip	The compression algorithm to use when publishing data to Kafka. Options are gzip, snappy, lz4, and none.
enable_auth	bool	false
auth_type	enum	basic	basic, sasl, or kerberos
basic_username	string
basic_password	string
sasl_username	string
sasl_password	enum
sasl_mechanism	string	SCRAM-SHA-256	SCRAM-SHA-256, SCRAM-SHA-512, or PLAIN
kerberos_service_name	string
kerberos_realm	string
kerberos_config_file	string	/etc/krb5.conf
kerberos_auth_type	enum	keytab	keytab or basic
kerberos_keytab_file	string	/etc/security/kafka.keytab
kerberos_username	string
kerberos_password	string

Supported Retry and Queuing Settings

This destination supports the retry settings, the sending queue settings, and the persistent queue settings.

Sending Queue	Persistent Queue	Retry on Failure
✓	✓	✓

Example Configuration

Basic Configuration

Web Interface

Standalone Destination

apiVersion: bindplane.observiq.com/v1
kind: Destination
metadata:
  name: kafka
spec:
  type: kafka_otlp_destination
  parameters:
    - name: telemetry_types
      value:
        - Metrics
        - Logs
        - Traces
    - name: protocol_version
      value: 2.0.0
    - name: brokers
      value:
        - localhost:3333
    - name: timeout
      value: 50
    - name: metric_topic
      value: my_metrics
    - name: metric_encoding
      value: otlp_json
    - name: log_topic
      value: my_logs
    - name: log_encoding
      value: raw
    - name: trace_topic
      value: my_spans
    - name: trace_encoding
      value: zipkin_proto
    - name: compression
      value: gzip