Middleware

Send logs, metrics, and traces to Middleware using the OpenTelemetry Protocol (OTLP) HTTP exporter.

Supported Telemetry Types

Metrics	Logs	Traces
✓	✓	✓

Prerequisites

Before configuring the Middleware destination, ensure you have:

• A running Middleware instance

• Your Middleware instance name

• A Middleware API key for authentication

• Network connectivity from your Bindplane agent to the Middleware instance

• At least one telemetry type selected (logs, metrics, or traces)

For help with Middleware setup, see the Middleware documentation.

Configuration

Parameter	Type	Default	Description	Required
Choose Telemetry Type	Selector	Logs, Metrics, Traces	Select which types of telemetry to export	Yes
Middleware Instance Name	String	-	Middleware instance name to send telemetry to	Yes
API Token	String	-	API Token for authenticating with the Middleware API	Yes

Advanced Configuration

General Settings

Settings for controlling log processing and timeout behavior.

Parameter	Type	Default	Description	Required
Drop Raw Copy	Boolean	true	Drop the raw copy of the log record stored in log.record.original	No
Timeout	Integer	30	Timeout in seconds for sending batches to Middleware	Yes

Retry and Queuing

This destination supports the retry settings, the sending queue settings, and the persistent queue settings.

Sending Queue	Persistent Queue	Retry on Failure
✓	✓	✓

Examples

Configuration Tips

Telemetry Type Selection

• All telemetry types (default): Export logs, metrics, and traces for comprehensive observability

• Custom selection: Choose only the telemetry types you need to optimize resource usage

• Flexible configuration: Can be changed at any time without redeploying

Instance Name and API Token

• Instance name must match your Middleware instance identifier

• API token provides secure authentication

• Keep API token secure; use environment variables or secret management systems

• Store credentials in a secure location and never commit them to version control

Raw Copy Handling

• Enable Drop Raw Copy to reduce storage for log telemetry

• Disable only if you need the original raw log record for debugging

• Default behavior (enabled) is recommended for production environments

Security Best Practices

• Store API tokens securely using environment variables or secret management systems

• Regularly rotate API tokens per your security policy

• Monitor token usage for suspicious activity

• Never share API tokens outside your organization

Performance Tuning

• Increase Queue Size for high-volume telemetry collection

• Adjust Number of Consumers based on available CPU and network bandwidth

• Enable Persistent Queuing for mission-critical data

• Enable Synchronize Persistent Queue to Disk in critical environments for data durability

Multi-Telemetry Considerations

• Each telemetry type is routed to its appropriate Middleware receiver automatically

• Compression is applied automatically to all telemetry types

• Retry and queue settings apply uniformly across all selected types

• Custom resource attributes can be added via resource processors in your configuration

Troubleshooting

Connection Refused or Timeout

Symptoms: Telemetry is not being sent; connection timeout errors in logs.

Solutions:

1. Verify Middleware instance is running and accessible

2. Verify the instance name is correct

3. Verify firewall rules allow outbound HTTPS traffic

4. Check that the API token is valid

Authentication Failures

Symptoms: "401 Unauthorized" or authentication errors in logs.

Solutions:

1. Verify the API token is correct and valid

2. Check that the token hasn't expired

3. Ensure the token has appropriate permissions for your Middleware instance

4. Verify both the instance name and API token are correctly configured

5. Check Middleware logs for authentication errors

Telemetry Not Appearing

Symptoms: Telemetry is sent but doesn't appear in Middleware UI or queries.

Solutions:

1. Verify the telemetry type is included in the configuration

2. Check that sources are generating telemetry

3. Verify proper routing and processing of telemetry data

4. Check Middleware ingestion logs for errors or rejections

5. Ensure the instance name matches the one used in Middleware

High Queue Depth

Symptoms: Sending queue is consistently full; telemetry may be dropped if persistent queuing is disabled.

Solutions:

1. Verify Middleware instance can handle the telemetry volume

2. Check for network latency or bandwidth issues

3. Increase Queue Size to buffer more telemetry

4. Increase Number of Consumers to process batches faster

5. Enable Persistent Queuing to prevent data loss during outages

6. Consider distributing telemetry across multiple Middleware instances

Performance Issues

Symptoms: High CPU or memory usage; slow telemetry ingestion.

Solutions:

1. Monitor Bindplane agent CPU and memory usage

2. Consider reducing telemetry volume at the source (sampling, filtering)

3. Disable Synchronize Persistent Queue to Disk if enabled (unless data integrity is critical)

4. Verify network connectivity and latency to Middleware

5. Consider batching at the source to reduce telemetry frequency

Data Loss

Symptoms: Telemetry is dropped after restarts or network outages.

Solutions:

1. Enable Persistent Queuing

2. Verify persistent queue directory exists and has write permissions

3. Monitor queue size to ensure it's not overflowing

4. Enable Retry on Failure to handle transient failures

5. Check available disk space for persistent queue storage

6. For critical environments, enable Synchronize Persistent Queue to Disk

7. Review logs for exporter errors or capacity issues

Related Resources

• Middleware Documentation

• Middleware OpenTelemetry Documentation

• OpenTelemetry Protocol Specification

• Bindplane Destinations Overview

• Retry Configuration

• Sending Queue Configuration

• Persistent Queue Configuration