# Middleware ### Supported Telemetry Types | Metrics | Logs | Traces | | ------- | ---- | ------ | | ✓ | ✓ | ✓ | ### Prerequisites Before configuring the Middleware destination, ensure you have: * A running Middleware instance * Your Middleware instance name * A Middleware API key for authentication * Network connectivity from your Bindplane agent to the Middleware instance * At least one telemetry type selected (logs, metrics, or traces) For help with Middleware setup, see the [Middleware documentation](https://docs.middleware.io/). ### Configuration | Parameter | Type | Default | Description | Required | | ------------------------ | -------- | --------------------------- | ---------------------------------------------------- | -------- | | Choose Telemetry Type | Selector | `Logs`, `Metrics`, `Traces` | Select which types of telemetry to export | Yes | | Middleware Instance Name | String | - | Middleware instance name to send telemetry to | Yes | | API Token | String | - | API Token for authenticating with the Middleware API | Yes | ### Advanced Configuration #### General Settings Settings for controlling log processing and timeout behavior. | Parameter | Type | Default | Description | Required | | ------------- | ------- | ------- | ------------------------------------------------------------------- | -------- | | Drop Raw Copy | Boolean | `true` | Drop the raw copy of the log record stored in `log.record.original` | No | | Timeout | Integer | `30` | Timeout in seconds for sending batches to Middleware | Yes | #### Retry and Queuing This destination supports the [retry settings](https://docs.bindplane.com/configuration/bindplane-otel-collector/retry-on-failure), the [sending queue settings](https://docs.bindplane.com/configuration/bindplane-otel-collector/sending-queue), and the [persistent queue settings.](https://docs.bindplane.com/configuration/bindplane-otel-collector/persistent-queue) | Sending Queue | Persistent Queue | Retry on Failure | | ------------- | ---------------- | ---------------- | | ✓ | ✓ | ✓ | ### Examples
### Configuration Tips #### Telemetry Type Selection * **All telemetry types** (default): Export logs, metrics, and traces for comprehensive observability * **Custom selection**: Choose only the telemetry types you need to optimize resource usage * **Flexible configuration**: Can be changed at any time without redeploying #### Instance Name and API Token * Instance name must match your Middleware instance identifier * API token provides secure authentication * Keep API token secure; use environment variables or secret management systems * Store credentials in a secure location and never commit them to version control #### Raw Copy Handling * Enable Drop Raw Copy to reduce storage for log telemetry * Disable only if you need the original raw log record for debugging * Default behavior (enabled) is recommended for production environments #### Security Best Practices * Store API tokens securely using environment variables or secret management systems * Regularly rotate API tokens per your security policy * Monitor token usage for suspicious activity * Never share API tokens outside your organization #### Performance Tuning * Increase Queue Size for high-volume telemetry collection * Adjust Number of Consumers based on available CPU and network bandwidth * Enable Persistent Queuing for mission-critical data * Enable Synchronize Persistent Queue to Disk in critical environments for data durability #### Multi-Telemetry Considerations * Each telemetry type is routed to its appropriate Middleware receiver automatically * Compression is applied automatically to all telemetry types * Retry and queue settings apply uniformly across all selected types * Custom resource attributes can be added via resource processors in your configuration ### Troubleshooting #### Connection Refused or Timeout **Symptoms**: Telemetry is not being sent; connection timeout errors in logs. **Solutions**: 1. Verify Middleware instance is running and accessible 2. Verify the instance name is correct 3. Verify firewall rules allow outbound HTTPS traffic 4. Check that the API token is valid #### Authentication Failures **Symptoms**: "401 Unauthorized" or authentication errors in logs. **Solutions**: 1. Verify the API token is correct and valid 2. Check that the token hasn't expired 3. Ensure the token has appropriate permissions for your Middleware instance 4. Verify both the instance name and API token are correctly configured 5. Check Middleware logs for authentication errors #### Telemetry Not Appearing **Symptoms**: Telemetry is sent but doesn't appear in Middleware UI or queries. **Solutions**: 1. Verify the telemetry type is included in the configuration 2. Check that sources are generating telemetry 3. Verify proper routing and processing of telemetry data 4. Check Middleware ingestion logs for errors or rejections 5. Ensure the instance name matches the one used in Middleware #### High Queue Depth **Symptoms**: Sending queue is consistently full; telemetry may be dropped if persistent queuing is disabled. **Solutions**: 1. Verify Middleware instance can handle the telemetry volume 2. Check for network latency or bandwidth issues 3. Increase Queue Size to buffer more telemetry 4. Increase Number of Consumers to process batches faster 5. Enable Persistent Queuing to prevent data loss during outages 6. Consider distributing telemetry across multiple Middleware instances #### Performance Issues **Symptoms**: High CPU or memory usage; slow telemetry ingestion. **Solutions**: 1. Monitor Bindplane agent CPU and memory usage 2. Consider reducing telemetry volume at the source (sampling, filtering) 3. Disable Synchronize Persistent Queue to Disk if enabled (unless data integrity is critical) 4. Verify network connectivity and latency to Middleware 5. Consider batching at the source to reduce telemetry frequency #### Data Loss **Symptoms**: Telemetry is dropped after restarts or network outages. **Solutions**: 1. Enable Persistent Queuing 2. Verify persistent queue directory exists and has write permissions 3. Monitor queue size to ensure it's not overflowing 4. Enable Retry on Failure to handle transient failures 5. Check available disk space for persistent queue storage 6. For critical environments, enable Synchronize Persistent Queue to Disk 7. Review logs for exporter errors or capacity issues ### Related Resources * [Middleware Documentation](https://docs.middleware.io/) * [Middleware OpenTelemetry Documentation](https://docs.middleware.io/open-telemetry/otel-getting-started) * [OpenTelemetry Protocol Specification](https://opentelemetry.io/docs/specs/otel/protocol/) * [Bindplane Destinations Overview](https://docs.bindplane.com/integrations/destinations) * [Retry Configuration](https://docs.bindplane.com/configuration/bindplane-otel-collector/retry-and-queueing) * [Sending Queue Configuration](https://docs.bindplane.com/configuration/bindplane-otel-collector/sending-queue) * [Persistent Queue Configuration](https://docs.bindplane.com/configuration/bindplane-otel-collector/persistent-queue)