> For the complete documentation index, see [llms.txt](https://docs.bindplane.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bindplane.com/integrations/destinations/elasticsearch-otlp.md). # Elasticsearch (OTLP) The Elasticsearch (OTLP) destination exports logs, metrics, and traces from a Bindplane pipeline to Elasticsearch by sending OTLP data to an Elastic APM Server. The OTLP/HTTP exporter is used for `Elastic Cloud` deployments, and the OTLP/gRPC exporter is used for `Self-Managed` deployments. Sending through the APM Server preserves the validation and data processing the APM Server performs, so data remains viewable in the Kibana Observability apps. Sending directly to Elasticsearch with the `elasticsearch` exporter bypasses that processing. ### Supported Telemetry | Metrics | Logs | Traces | | ------- | ---- | ------ | | ✓ | ✓ | ✓ | Use **Choose Telemetry Type** to limit which signals this destination exports. By default all three are enabled. ### Prerequisites You need a reachable Elastic APM Server OTLP endpoint and credentials to authenticate against it. * **An Elastic deployment with an APM Server.** Either Elastic Cloud or a Self-Managed Elastic Stack that exposes the APM Server OTLP intake. * **The APM Server endpoint.** For Elastic Cloud, the APM Server URL. For Self-Managed, the hostname or IP and the OTLP port (default `8200`). * **An APM Secret Token (or API key).** Sent as an `Authorization: Bearer ` header. In Elastic Cloud the secret token is set on the Elastic APM integration in the agent policy. APM Server receives traces, metrics, and logs over OTLP using OTLP/HTTP (`/v1/logs`, `/v1/metrics`, `/v1/traces`) and OTLP/gRPC on the same port. See Elastic's [Use OpenTelemetry with Elastic APM](https://www.elastic.co/docs/solutions/observability/apm/opentelemetry) and the [OpenTelemetry intake API](https://www.elastic.co/docs/solutions/observability/apm/opentelemetry-intake-api) for endpoint and authentication details. #### How to find your APM Server URL and Secret Token **Elastic Cloud** * Navigate to your Elastic deployment. * Go to **Management > Fleet > Agent Policies** (search for *agent policies*). * Select the Agent Policy you want to configure. If none exists, create one. * Under the integrations tab, find the **Elastic APM** row, open its actions menu, and select **Edit Integration**. * The Server URL is under **General > Server Configuration > URL**. * The Secret Token is under **Agent Authorization > Secret token**, and can be set here if needed. **Self-Managed** For Kubernetes-hosted Elastic, see Elastic's [Connect to the APM Server](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-connecting.html). ### Configuration
Bindplane docs - Elasticsearch (OTLP) - image 1
**Connection** | Parameter | Type | Required | Default | Description | | -------------------------------- | --------------------------------- | -------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | Choose Telemetry Type | Telemetry Selector | No | Logs, Metrics, Traces | Which signals this destination exports. | | Elastic Instance Deployment Type | Enum: Self-Managed, Elastic Cloud | No | Elastic Cloud | The deployment model of your Elastic instance. Selects the OTLP/HTTP exporter (`Elastic Cloud`) or the OTLP/gRPC exporter (`Self-Managed`). | | APM Server URL | String | Yes | *(empty)* | The URL of your Elastic APM Server. Relevant when Deployment Type is `Elastic Cloud`. | | Hostname | String | Yes | *(empty)* | The hostname or IP address of your Elastic APM Server. Relevant when Deployment Type is `Self-Managed`. | | Port | Integer | No | 8200 | TCP port to which the exporter sends OTLP data. Relevant when Deployment Type is `Self-Managed`. | | Secret Token | String (sensitive) | Yes | *(empty)* | Elastic APM Secret Token, sent as an `Authorization: Bearer` header. | **TLS** These parameters are relevant when Deployment Type is `Self-Managed`. `Elastic Cloud` deployments always use TLS. | Parameter | Type | Required | Default | Description | | --------------------------------- | ------- | -------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | Enable TLS | Boolean | No | true | Configure advanced TLS settings. | | Skip TLS Certificate Verification | Boolean | No | false | Skip TLS certificate verification. Relevant when Enable TLS is `true`. | | TLS Certificate Authority File | String | No | *(empty)* | Certificate authority used to validate TLS certificates. Relevant when Enable TLS is `true` and Skip TLS Certificate Verification is `false`. | | Server Name Override | String | No | *(empty)* | Optional virtual hostname indicating the server name requested by the client. Generally not required. Relevant when Enable TLS is `true`. | | Mutual TLS | Boolean | No | false | Use mutual TLS authentication. Relevant when Enable TLS is `true`. | | TLS Client Certificate File | String | No | *(empty)* | A TLS certificate used for client authentication. Relevant when Enable TLS and Mutual TLS are `true`. | | TLS Client Private Key File | String | No | *(empty)* | A TLS private key used for client authentication. Relevant when Enable TLS and Mutual TLS are `true`. | **Advanced** | Parameter | Type | Required | Default | Description | | ------------------ | ---------------------- | -------- | --------- | --------------------------------------------------------------------------------- | | Compression | Enum: none, gzip, zlib | No | gzip | Compression algorithm to use when sending data to the APM Server. | | Additional Headers | Map | No | *(empty)* | Additional headers to attach to each request. | | Drop Raw Copy | Boolean | No | true | When enabled, the raw copy of the log stored in `log.record.original` is dropped. | **Retry on Failure** | Parameter | Type | Required | Default | Description | | ----------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | Enable Retry on Failure | Boolean | No | true | Resend telemetry data that failed to transmit to the destination. | | Initial interval | Integer | No | 5 | Time (in seconds) to wait after the first failure before retrying. Relevant when Enable Retry on Failure is `true`. | | Max interval | Integer | No | 30 | The upper bound (in seconds) on backoff. Relevant when Enable Retry on Failure is `true`. | | Max elapsed time | Integer | No | 300 | The maximum time (in seconds) spent trying to send a batch, to avoid a never-ending retry loop. Relevant when Enable Retry on Failure is `true`. | **Sending Queue** | Parameter | Type | Required | Default | Description | | ------------------------- | --------- | -------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Enable Sending Queue | Boolean | No | true | Buffer telemetry data temporarily before sending to help avoid data loss during a temporary network outage. | | Number of Consumers | Integer | No | 10 | Number of consumers that dequeue batches. Relevant when Enable Sending Queue is `true`. | | Queue Size | Integer | No | 5000 | Maximum number of batches kept in memory before dropping. Relevant when Enable Sending Queue is `true`. | | Enable Persistent Queuing | Boolean | No | true | Buffer telemetry data to disk before sending to help avoid data loss during network outages or collector restarts. Relevant when Enable Sending Queue is `true`. | | Persistent Queue Storage | Extension | Yes | `file_storage_persistent_queue` | The storage extension used for the persistent queue. Relevant when Enable Sending Queue and Enable Persistent Queuing are `true`. | ### Examples #### Send all signals to an Elastic Cloud APM Server This destination exports logs, metrics, and traces to an Elastic Cloud APM Server using the OTLP/HTTP exporter with gzip compression. ```yaml apiVersion: bindplane.observiq.com/v1 kind: Destination metadata: name: elasticsearch-otlp spec: type: elasticsearch_otlp parameters: - name: telemetry_types value: - Logs - Metrics - Traces - name: deployment_type value: Elastic Cloud - name: server_url value: https://my-deployment.apm.us-east-1.aws.cloud.es.io:443 - name: secret_token value: REPLACE_WITH_SECRET_TOKEN - name: compression value: gzip ``` ### Configuration Tips * Choose the deployment type that matches your endpoint. `Elastic Cloud` uses OTLP/HTTP against the APM Server URL, while `Self-Managed` uses OTLP/gRPC against the hostname and port (default `8200`). * Leave **Drop Raw Copy** enabled to avoid storing a duplicate of each log under `log.record.original`. Disable it only when you need the original record retained downstream. * Keep the sending queue and persistent queue enabled for resilience. The persistent queue buffers to disk, so data survives collector restarts and short network outages. ### Troubleshooting #### Authentication fails Symptoms: requests are rejected with 401/403 (Elastic Cloud) or `Unauthenticated` (Self-Managed); no data appears in Kibana. Solutions: 1. Verify the Secret Token matches the APM integration's secret token in the agent policy. 2. Confirm the token is current and the APM integration is enabled on the policy. #### Connection or TLS errors Symptoms: connection refused, timeouts, or certificate verification failures. Solutions: 1. Confirm the APM Server URL (Elastic Cloud) or hostname and port (Self-Managed) are correct and reachable from the collector. 2. For Self-Managed with a private CA, set the TLS Certificate Authority File. Only set Skip TLS Certificate Verification for testing. #### Data appears throttled or dropped Symptoms: gaps in data, exporter retry or queue-full warnings in collector logs. Solutions: 1. Keep Retry on Failure enabled and tune the initial and max intervals. 2. Increase the Queue Size or number of consumers, and enable persistent queuing so data is not lost when the in-memory queue fills. ### Standalone Destination ```yaml apiVersion: bindplane.observiq.com/v1 kind: Destination metadata: name: elasticsearch-otlp spec: type: elasticsearch_otlp parameters: - name: telemetry_types value: - Logs - Metrics - Traces - name: deployment_type value: Elastic Cloud - name: server_url value: https://my-deployment.apm.us-east-1.aws.cloud.es.io:443 - name: secret_token value: REPLACE_WITH_SECRET_TOKEN ``` ### Related Resources * [OTLP/HTTP Exporter](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/README.md) (used for `Elastic Cloud`) * [OTLP Exporter](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlpexporter/README.md) (used for `Self-Managed`) * [Use OpenTelemetry with Elastic APM](https://www.elastic.co/docs/solutions/observability/apm/opentelemetry) * [OpenTelemetry intake API](https://www.elastic.co/docs/solutions/observability/apm/opentelemetry-intake-api) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.bindplane.com/integrations/destinations/elasticsearch-otlp.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.