Splunk (HEC)

The Splunk (HEC) destination sends logs and metrics to a Splunk HTTP Event Collector (HEC) endpoint. HEC is an HTTP/HTTPS listener on the Splunk indexer or Cloud Platform stack that accepts event and metric data authenticated with a token.

Supported Telemetry Types

Platform	Metrics	Logs	Traces
Linux	✓	✓
Windows	✓	✓
macOS	✓	✓

Prerequisites

• A Splunk HTTP Event Collector enabled on your Splunk indexer or Splunk Cloud Platform stack.
• A valid HEC authentication token. See Set up and use HTTP Event Collector.
• Network access from the collector to the HEC endpoint (default TCP port 8088).
• Optionally, the name of a Splunk index for logs and/or metrics. Metric data must be sent to a metrics-type index.

Configuration

General

Parameter	Type	Default	Description
Choose Telemetry Type	Telemetry Selector	["Logs", "Metrics"]	Which telemetry types to export. Valid values are Logs and Metrics.
Hostname	String	localhost	Hostname or IP address of the HTTP Event Collector.
Port	Integer	8088	TCP port to which the exporter sends data. Valid range is 1–65535.
Path	String	/services/collector/event	The HTTP API path to which the exporter sends data.
Token	String		Authentication token used when connecting to the HTTP Event Collector. Required. Sensitive (stored as a secret).

Logs

These parameters apply when telemetry_types includes Logs.

Parameter	Type	Default	Description
Index	String		Optional name of the Splunk index targeted for logs.
Max Request Size	Integer	2097152	The maximum size (in bytes) of a log request sent to the destination.
Profiling Data	Boolean	true	Send profiling data to Splunk.
Export Raw	Boolean	false	Send only the log's body.

Metrics

These parameters apply when telemetry_types includes Metrics.

Parameter	Type	Default	Description
Index	String		Optional name of the Splunk index targeted for metrics.
Max Request Size	Integer	2097152	The maximum size (in bytes) of a metric request sent to the destination.
Use Multi-Metric Format	Boolean	false	Combines metrics with the same metadata to reduce ingest. See The multiple-metric JSON format.

Advanced

Parameter	Type	Default	Description
Max Event Size	Integer	5242880	The maximum size (in bytes) of an individual event. Events larger than this are dropped.
Enable Compression	Boolean	true	Compress telemetry data using gzip before sending.
Source	String		The source value to use for all telemetry sent to Splunk.
Source Type	String		The sourcetype value to use for all telemetry sent to Splunk.
Drop Raw Copy	Boolean	true	When enabled, the raw copy of the log stored in log.record.original is dropped.

TLS

These parameters apply when enable_tls is true.

Parameter	Type	Default	Description
Enable TLS	Boolean	false	Whether or not to use TLS.
Skip TLS Certificate Verification	Boolean	false	Enable to skip TLS certificate verification.
TLS Certificate Authority File	String		Certificate authority used to validate TLS certificates.

Retry on Failure

Parameter	Type	Default	Description
Enable Retry on Failure	Boolean	true	Attempt to resend telemetry data that has failed to transmit.
Initial interval	Integer	5	Time (in seconds) to wait after the first failure before retrying.
Max interval	Integer	30	The upper bound (in seconds) on backoff.
Max elapsed time	Integer	300	The maximum time (in seconds) spent trying to send a batch before giving up.

Sending Queue

Parameter	Type	Default	Description
Enable Sending Queue	Boolean	true	Buffer telemetry data temporarily before sending to avoid loss during temporary network outages.
Number of Consumers	Integer	10	Number of consumers that dequeue batches.
Queue Size	Integer	5000	Maximum number of batches kept in memory before dropping.
Enable Persistent Queuing	Boolean	true	Buffer telemetry data to disk to avoid loss during network outages or collector restarts.
Persistent Queue Storage	Extension	file_storage_persistent_queue	The storage to use for the persistent queue. See Persistent Queue.

Example Configuration

Standalone Destination

apiVersion: bindplane.observiq.com/v1
kind: Destination
metadata:
  id: splunkhec_v2
  name: splunkhec_v2
spec:
  type: splunkhec_v2
  parameters:
    - name: telemetry_types
      value: ['Logs', 'Metrics']
    - name: hostname
      value: 'splunk.example.com'
    - name: port
      value: '8088'
    - name: path
      value: '/services/collector/event'
    - name: token
      value: '00000000-0000-0000-0000-000000000000'
    - name: index_logs
      value: 'main'
    - name: index_metrics
      value: 'metrics'
    - name: enable_tls
      value: 'true'
    - name: ca_file
      value: '/opt/observiq-otel-collector/splunk-ca.crt'

Configuration Tips

Indexes and sourcetype

• Set index_logs and index_metrics to route data to specific Splunk indexes. Leave them empty to use the index the HEC token is configured to default to. Metrics must target a metrics-type index.
• Use source and sourcetype to label all telemetry from this destination so it can be searched and routed in Splunk.

TLS

• Splunk HEC commonly runs over HTTPS. Set enable_tls to true and provide a ca_file so the collector can validate the HEC certificate.
• Use insecure_skip_verify only for testing against an endpoint with a self-signed or untrusted certificate. Avoid it in production.

Throughput

• Leave enable_compression on (the default) to reduce bandwidth to the HEC endpoint.
• For high metric volume, enable use_multi_metric_format to combine metrics that share metadata into fewer events and reduce ingest.

Troubleshooting

Symptom: HEC rejects requests with a 403 or "Invalid token" error. Solution: Confirm the token matches an enabled HEC token in Splunk and that HEC is enabled on the stack. The token is stored as a secret in Bindplane, so re-enter it if you are unsure it was saved correctly.

Symptom: TLS handshake or certificate verification failures. Solution: When enable_tls is true, provide a ca_file that signs the HEC endpoint's certificate. For testing only, you can set insecure_skip_verify to true. Verify hostname and port match the HEC listener (default 8088).

Symptom: Metrics are accepted but do not appear, or return an index error. Solution: Ensure index_metrics points to a metrics-type Splunk index. Sending metrics to an events index causes Splunk to reject or misfile the data.

Related Resources

• Set up and use HTTP Event Collector
• The multiple-metric JSON format
• Splexicon: source
• Splexicon: sourcetype
• Persistent Queue configuration