Honeycomb Enhance Indexing S3

Export logs and traces to AWS S3 with automatic field indexing for efficient rehydration and querying in Honeycomb.

This is only for the Honeycomb License Plan

Supported Telemetry Types

Telemetry Type	Supported
Traces	✓
Metrics
Logs	✓

Prerequisites

• A Honeycomb account with a Management API key and secret (must have enhance:write scope)

• An AWS S3 bucket with appropriate write permissions

• AWS credentials configured on the collector host

• Network connectivity from the collector to both S3 and the Honeycomb API endpoint

Configuration

Basic Configuration

The minimum configuration requires Honeycomb API credentials, an S3 bucket, and an AWS region.

Parameter	Type	Default	Description	Required
Telemetry Types	Selector	Logs, Traces	Select which types of telemetry to export.	Yes
API Key	String		Honeycomb Management API key with enhance:write scope.	Yes
API Secret	String		Honeycomb Management API secret.	Yes
API Endpoint	String	https://api.honeycomb.io	Honeycomb API endpoint URL.	Yes
Region	Enum	us-east-1	AWS region where the S3 bucket is located.	Yes
Bucket	String		Name of the S3 bucket to export telemetry into.	Yes
Folder Prefix	String		Root directory of the bucket to export telemetry into.	No
Partition Format	String	year=%Y/month=%m/day=%d/hour=%H/minute=%M	Time-based partition format using ctime format codes. Must contain year, month, day, hour, and minute placeholders.	Yes

---

Advanced Configuration

Data Format and Compression

Parameter	Type	Default	Description	Required
Data Format	Enum	otlp_proto	Format of telemetry data written to S3. Options: otlp_proto (smaller files, more efficient) or otlp_json (human-readable).	No
Compression	Enum	gzip	Compression applied to data stored in S3. Options: gzip or none.	No

AWS Credentials

Parameter	Type	Default	Description	Required
Role ARN	String		IAM Role ARN to assume for S3 access. If not set, the collector uses the default AWS SDK credential chain (environment variables, IAM instance profile, shared credentials file).	No

Note: This destination does not accept inline AWS access keys or secret keys. The collector resolves credentials using the standard AWS SDK credential chain. You can optionally specify a Role ARN to assume a cross-account or scoped IAM role. For all other credential methods (environment variables, instance profiles, shared credentials files), configure them on the collector host directly.

S3 Connection Options

Parameter	Type	Default	Description	Required
Endpoint	String		Overrides the S3 endpoint URL. Use this for S3-compatible services like MinIO.	No
Force Path Style	Boolean	false	Force path-style S3 addressing instead of virtual-host style. Required for some S3-compatible services.	No
Disable SSL	Boolean	false	Disable SSL verification for S3 connections.	No

Indexing

Parameter	Type	Default	Description	Required
Custom Indexed Fields	Strings		Additional fields to index beyond the automatic ones (trace.trace_id, service.name, session.id).	No

Note: To minimize Collector processing compute and time, Honeycomb recommends indexing no more than 5 custom fields.

General Settings

Parameter	Type	Default	Description	Required
Drop Raw Copy	Boolean	true	Drop the raw copy of the log record stored in log.record.original.	No
Timeout	Integer	30	Maximum time in seconds to wait for a batch to be sent.	No
Retry Mode	Enum	standard	S3 client retry strategy. Options: standard or adaptive.	No
Maximum Retries	Integer	3	Maximum number of retry attempts for S3 uploads. Set to 0 for unlimited retries.	No

Retry and Queuing

This destination supports the retry settings, the sending queue settings, and the persistent queue settings.

---

Configuration Tips

S3 Bucket and Credentials

• AWS credentials are resolved via the standard AWS SDK credential chain (IAM role, environment variables, or shared credentials file). This destination does not accept inline AWS access keys.

• To use a specific IAM role (e.g., for cross-account access), set the Role ARN field in Advanced Configuration. The collector will assume this role using STS.

• Make sure the S3 bucket policy grants s3:PutObject permission to the collector's identity (or the assumed role).

• Use the Folder Prefix to isolate telemetry data within a shared bucket.

Custom Indexed Fields

• Fields like user.id or environment can be added to Custom Indexed Fields for faster rehydration queries in Honeycomb.

• Each additional indexed field increases S3 storage usage due to extra index files written alongside data files.

Honeycomb API Endpoint

• Use https://api.honeycomb.io for US-based Honeycomb accounts.

• Use https://api.eu1.honeycomb.io for EU-based Honeycomb accounts.

---

Troubleshooting

S3 Upload Failures

Errors referencing AccessDenied, NoSuchBucket, or InvalidAccessKeyId in collector logs.

1. Verify the S3 bucket exists in the configured region and the collector host has valid AWS credentials with s3:PutObject permission.

2. If using a custom Endpoint for S3-compatible storage, make sure Force Path Style is enabled.

Honeycomb API Authentication Errors

Errors referencing 401 Unauthorized or 403 Forbidden from the Honeycomb API endpoint.

1. Confirm the API Key and API Secret are for a Management API key with the enhance:write scope. Standard ingest keys will not work.

2. Verify the API Endpoint matches your Honeycomb account region (US vs EU).

Missing or Incomplete Index Files

Data appears in S3 but index files are missing or incomplete. Rehydration queries in Honeycomb return fewer results than expected.

1. Check that the partition format contains all required placeholders (%Y, %m, %d, %H, %M). Missing placeholders will cause validation failures.

2. Verify that custom indexed field names match the exact attribute keys present in your telemetry data.

---

Related Resources

• Honeycomb: Collector Export to S3 Archive

• Honeycomb: Managing API Keys

• Enhance Indexing S3 Exporter (GitHub)

• Bindplane Destinations Overview