# Honeycomb Enhance Indexing S3 {% hint style="info" %} This is only for the Honeycomb License Plan {% endhint %} ### Supported Telemetry Types | Telemetry Type | Supported | | -------------- | --------- | | Traces | ✓ | | Metrics | | | Logs | ✓ | ### Prerequisites * A Honeycomb account with a Management API key and secret (must have `enhance:write` scope) * An AWS S3 bucket with appropriate write permissions * AWS credentials configured on the collector host * Network connectivity from the collector to both S3 and the Honeycomb API endpoint ### Configuration #### Basic Configuration The minimum configuration requires Honeycomb API credentials, an S3 bucket, and an AWS region. | Parameter | Type | Default | Description | Required | | ---------------- | -------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -------- | | Telemetry Types | Selector | `Logs`, `Traces` | Select which types of telemetry to export. | Yes | | API Key | String | | Honeycomb Management API key with `enhance:write` scope. | Yes | | API Secret | String | | Honeycomb Management API secret. | Yes | | API Endpoint | String | `https://api.honeycomb.io` | Honeycomb API endpoint URL. | Yes | | Region | Enum | `us-east-1` | AWS region where the S3 bucket is located. | Yes | | Bucket | String | | Name of the S3 bucket to export telemetry into. | Yes | | Folder Prefix | String | | Root directory of the bucket to export telemetry into. | No | | Partition Format | String | `year=%Y/month=%m/day=%d/hour=%H/minute=%M` | Time-based partition format using ctime format codes. Must contain year, month, day, hour, and minute placeholders. | Yes | *** #### Advanced Configuration **Data Format and Compression** | Parameter | Type | Default | Description | Required | | ----------- | ---- | ------------ | ------------------------------------------------------------------------------------------------------------------------------ | -------- | | Data Format | Enum | `otlp_proto` | Format of telemetry data written to S3. Options: `otlp_proto` (smaller files, more efficient) or `otlp_json` (human-readable). | No | | Compression | Enum | `gzip` | Compression applied to data stored in S3. Options: `gzip` or `none`. | No | **AWS Credentials** | Parameter | Type | Default | Description | Required | | --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | | Role ARN | String | | IAM Role ARN to assume for S3 access. If not set, the collector uses the default AWS SDK credential chain (environment variables, IAM instance profile, shared credentials file). | No | > **Note:** This destination does not accept inline AWS access keys or secret keys. The collector resolves credentials using the [standard AWS SDK credential chain](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials). You can optionally specify a Role ARN to assume a cross-account or scoped IAM role. For all other credential methods (environment variables, instance profiles, shared credentials files), configure them on the collector host directly. **S3 Connection Options** | Parameter | Type | Default | Description | Required | | ---------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------- | -------- | | Endpoint | String | | Overrides the S3 endpoint URL. Use this for S3-compatible services like MinIO. | No | | Force Path Style | Boolean | `false` | Force path-style S3 addressing instead of virtual-host style. Required for some S3-compatible services. | No | | Disable SSL | Boolean | `false` | Disable SSL verification for S3 connections. | No | **Indexing** | Parameter | Type | Default | Description | Required | | --------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------ | -------- | | Custom Indexed Fields | Strings | | Additional fields to index beyond the automatic ones (`trace.trace_id`, `service.name`, `session.id`). | No | > **Note:** To minimize Collector processing compute and time, Honeycomb recommends indexing no more than 5 custom fields. **General Settings** | Parameter | Type | Default | Description | Required | | --------------- | ------- | ---------- | ---------------------------------------------------------------------------------- | -------- | | Drop Raw Copy | Boolean | `true` | Drop the raw copy of the log record stored in `log.record.original`. | No | | Timeout | Integer | `30` | Maximum time in seconds to wait for a batch to be sent. | No | | Retry Mode | Enum | `standard` | S3 client retry strategy. Options: `standard` or `adaptive`. | No | | Maximum Retries | Integer | `3` | Maximum number of retry attempts for S3 uploads. Set to `0` for unlimited retries. | No | **Retry and Queuing** This destination supports the [retry settings](https://docs.bindplane.com/configuration/bindplane-otel-collector/retry-on-failure), the [sending queue settings](https://docs.bindplane.com/configuration/bindplane-otel-collector/sending-queue), and the [persistent queue settings](https://docs.bindplane.com/configuration/bindplane-otel-collector/persistent-queue). *** ### Configuration Tips #### S3 Bucket and Credentials * AWS credentials are resolved via the standard AWS SDK credential chain (IAM role, environment variables, or shared credentials file). This destination does not accept inline AWS access keys. * To use a specific IAM role (e.g., for cross-account access), set the **Role ARN** field in Advanced Configuration. The collector will assume this role using STS. * Make sure the S3 bucket policy grants `s3:PutObject` permission to the collector's identity (or the assumed role). * Use the Folder Prefix to isolate telemetry data within a shared bucket. #### Custom Indexed Fields * Fields like `user.id` or `environment` can be added to Custom Indexed Fields for faster rehydration queries in Honeycomb. * Each additional indexed field increases S3 storage usage due to extra index files written alongside data files. #### Honeycomb API Endpoint * Use `https://api.honeycomb.io` for US-based Honeycomb accounts. * Use `https://api.eu1.honeycomb.io` for EU-based Honeycomb accounts. *** ### Troubleshooting #### S3 Upload Failures Errors referencing `AccessDenied`, `NoSuchBucket`, or `InvalidAccessKeyId` in collector logs. 1. Verify the S3 bucket exists in the configured region and the collector host has valid AWS credentials with `s3:PutObject` permission. 2. If using a custom Endpoint for S3-compatible storage, make sure Force Path Style is enabled. #### Honeycomb API Authentication Errors Errors referencing `401 Unauthorized` or `403 Forbidden` from the Honeycomb API endpoint. 1. Confirm the API Key and API Secret are for a Management API key with the `enhance:write` scope. Standard ingest keys will not work. 2. Verify the API Endpoint matches your Honeycomb account region (US vs EU). #### Missing or Incomplete Index Files Data appears in S3 but index files are missing or incomplete. Rehydration queries in Honeycomb return fewer results than expected. 1. Check that the partition format contains all required placeholders (`%Y`, `%m`, `%d`, `%H`, `%M`). Missing placeholders will cause validation failures. 2. Verify that custom indexed field names match the exact attribute keys present in your telemetry data. *** ### Related Resources * [Honeycomb: Collector Export to S3 Archive](https://docs.honeycomb.io/send-data/telemetry-pipeline/enhance/collector-export-s3/) * [Honeycomb: Managing API Keys](https://docs.honeycomb.io/configure/teams/manage-api-keys/) * [Enhance Indexing S3 Exporter (GitHub)](https://github.com/honeycombio/enhance-indexing-s3-exporter) * [Bindplane Destinations Overview](https://docs.bindplane.com/resources/destinations/) --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.bindplane.com/integrations/destinations/honeycomb-enhance-indexing-s3.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.