AWS S3

The AWS S3 destination exports telemetry from a Bindplane pipeline as objects written to an Amazon S3 bucket. It supports logs, metrics, and traces, writing each signal as timestamped, partitioned files under a configurable prefix using the OpenTelemetry awss3exporter.

Supported Telemetry Types

Metrics	Logs	Traces
✓	✓	✓

Prerequisites

• A pre-created S3 bucket in the AWS account and region you intend to export to. The exporter does not create the bucket.
• The AWS region the bucket resides in.
• AWS credentials available to the collector with permission to write objects to that bucket. At minimum the credentials need the s3:PutObject action on the bucket's objects (arn:aws:s3:::<bucket>/*). See the AWS identity-based policy examples for Amazon S3 and the PutObject API reference.

A minimal IAM policy granting object writes to a single bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BindplaneS3Export",
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-telemetry-bucket/*"
    }
  ]
}

If objects are encrypted with SSE-KMS, the credentials also need kms:GenerateDataKey and kms:Decrypt on the key.

Providing credentials to the collector

The exporter resolves credentials using the standard AWS SDK credential chain. Configure them on the host running the collector with one of:

• The aws CLI (AWS CLI getting started). The collector runs as root by default, so install and configure the CLI under the collector's system account.
• AWS environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION), for example via a systemd override.
• An instance role / IRSA when running on EC2, ECS, or EKS.

Configuration

Bucket and routing

Parameter	Type	Required	Default	Description
Telemetry Types	Telemetry Selector	No	Logs, Metrics, Traces	Which signals this destination exports.
Region	Enum (creatable)	Yes	us-east-1	The AWS region of the bucket. Choose from the listed regions or enter another valid region string.
Bucket	String	Yes	(empty)	Name of the S3 bucket to export telemetry into.
Folder Prefix	String	No	(empty)	Root directory within the bucket to export telemetry into.
File Prefix	String	No	(empty)	Prefix for the name of exported telemetry files.

Object format and storage (Advanced)

Parameter	Type	Required	Default	Description
Granularity	Enum: minute, hour	No	minute	Granularity of the timestamps in the S3 key. Used with AWS S3 exporter versions before v0.121.0; v0.121.0 and later use Partition Format instead.
Partition Format	String	No	year=%Y/month=%m/day=%d/hour=%H/minute=%M	strftime format defining the S3 key partition. Available with the AWS S3 exporter v0.121.0 and newer. See the strftime reference.
Compression	Enum: none, gzip	No	none	Whether data is sent and stored in S3 using gzip compression.
Storage Class	Enum	No	STANDARD	S3 storage class for written objects.1
Endpoint	String	No	(empty)	Overrides the endpoint used to connect to S3 instead of constructing it from the region and bucket. Generally not needed.
Resource Label to Prefix	String	No	com.awss3.prefix	An S3 prefix dynamically derived from the named resource attribute's value.

1. One of STANDARD, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, EXPRESS_ONEZONE, GLACIER_IR, GLACIER, DEEP_ARCHIVE. Available in agents v1.71.5 and later. See S3 storage classes.

Marshalers (Advanced)

The marshaler determines the on-disk format of exported objects. Defaults to otlp_json for every signal.

Parameter	Type	Required	Default	Description
Drop Raw Copy	Boolean	No	true	When enabled, drops the raw copy of the log stored in log.record.original before export. Applies to logs.
Logs Marshaler	Enum: otlp_json, otlp_proto, sumo_ic, body	No	otlp_json	Format of log data sent to S3. sumo_ic and body are logs-only. Shown when Logs is selected.
Metrics Marshaler	Enum: otlp_json, otlp_proto	No	otlp_json	Format of metric data sent to S3. Shown when Metrics is selected.
Traces Marshaler	Enum: otlp_json, otlp_proto	No	otlp_json	Format of trace data sent to S3. Shown when Traces is selected.

Timeout and Retry (Advanced)

Parameter	Type	Required	Default	Description
Timeout	Integer (seconds)	No	30	Maximum time to wait for a batch to be sent to the destination.
Retry Mode	Enum: standard, adaptive, nop	No	standard	The retryer implementation. Set to nop to disable retries.
Maximum Retries	Integer	No	3	Maximum number of times to retry a request. 0 retries all retryable errors until success or a non-retryable error. Shown when Retry Mode is not nop.
Maximum Backoff	Integer (seconds)	No	20	Maximum time to wait between retries. Shown when Retry Mode is standard.

Sending Queue (Advanced)

Parameter	Type	Required	Default	Description
Enable Sending Queue	Boolean	No	false	Buffer telemetry temporarily before sending to reduce loss during a temporary network outage. See the exporterhelper reference.
Number of Consumers	Integer	No	10	Number of consumers that dequeue batches. Shown when the sending queue is enabled.
Queue Size	Integer	No	5000	Maximum number of batches the queue can hold. Shown when the sending queue is enabled.
Wait for Result	Boolean	No	false	Block incoming requests until the request is processed. Not compatible with persistent queuing. Shown when the sending queue is enabled.
Block on Overflow	Boolean	No	false	If true, block the request until the queue has space; otherwise reject the data immediately. Shown when the sending queue is enabled.
Enable Persistent Queuing	Boolean	No	true	Buffer telemetry to disk so it survives network outages and collector restarts. Shown when the sending queue is enabled and Wait for Result is false.
Persistent Queue Storage	Extension	Yes	File storage (${OIQ_OTEL_COLLECTOR_HOME}/storage)	Storage extension backing the persistent queue. See persistent queue. Shown when the sending queue and persistent queuing are enabled and Wait for Result is false.

Examples

Export all signals to a partitioned bucket

This destination writes logs, metrics, and traces to the my-telemetry-bucket bucket in us-east-1, under a bindplane/ prefix, using gzip compression.

apiVersion: bindplane.observiq.com/v1
kind: Destination
metadata:
  name: aws-s3
spec:
  type: aws_s3
  parameters:
    - name: telemetry_types
      value:
        - Logs
        - Metrics
        - Traces
    - name: region
      value: us-east-1
    - name: bucket
      value: my-telemetry-bucket
    - name: prefix
      value: bindplane
    - name: compression
      value: gzip

Configuration Tips

• Set Region to the region the bucket actually lives in. A region mismatch produces authorization or endpoint errors at export time.
• Use gzip Compression and a colder Storage Class (for example STANDARD_IA or GLACIER_IR) to lower storage cost when objects are written for archival rather than frequent reads.
• For lossless delivery across collector restarts, keep the sending queue and persistent queuing enabled (the default). Wait for Result is incompatible with persistent queuing, so leave it off when persistence matters.

Troubleshooting

Access denied writing objects

Symptoms: the collector logs AccessDenied or 403 errors from S3 and no objects appear.

Solutions:

1. Confirm the credentials the collector resolves have s3:PutObject on arn:aws:s3:::<bucket>/*.
2. If the bucket enforces SSE-KMS, grant kms:GenerateDataKey and kms:Decrypt on the key.
3. Verify the credentials are installed under the account the collector runs as (root by default).

No objects in the bucket / wrong region

Symptoms: exports appear to succeed but the bucket is empty, or requests fail with endpoint errors.

Solutions:

1. Confirm Region matches the bucket's region.
2. Confirm the Bucket name is exact and exists in that account.
3. If an Endpoint override is set, confirm it is correct, otherwise clear it to let the region and bucket construct the endpoint.

Data dropped during outages

Symptoms: telemetry gaps coincide with network interruptions or collector restarts.

Solutions:

1. Enable the sending queue and persistent queuing so batches buffer to disk and replay on recovery.
2. Increase Queue Size if bursts overflow the queue.

Standalone Destination

apiVersion: bindplane.observiq.com/v1
kind: Destination
metadata:
  name: aws-s3
spec:
  type: aws_s3
  parameters:
    - name: telemetry_types
      value:
        - Logs
        - Metrics
        - Traces
    - name: region
      value: us-east-1
    - name: bucket
      value: my-telemetry-bucket

Related Resources

• awss3exporter — OpenTelemetry Collector Contrib reference
• Identity-based policy examples for Amazon S3