AWS S3 Rehydration

Object Format

Rehydrated data must be in OTLP format JSON for correct processing. Ensure your data adheres to this format.

The AWS S3 Destination supports gzip compression. This receiver will gracefully detect and handle if any data is gzipped or uncompressed.

Supported Platforms

Platform	Supported
Linux	✓
Windows	✓
macOS	✓
Kubernetes Cluster*	✓

Available in the Bindplane Distro for OpenTelemetry Collector v1.69.0+.

* Note: When deploying with a Kubernetes Cluster agent, select StatefulSet as the deployment method.

Configuration Fields

Field	Description
Region	The AWS recognized region string
Bucket	Name of the S3 Bucket to rehydration telemetry from.
Folder Prefix	Root directory of the bucket to rehydration telemetry from.
Poll Size	The max number of object descriptions to be returned by a single poll against the S3 API.
Batch Size	The max number of object descriptions to process & rehydrate at once after retrieving from the S3 API.
Starting Time	UTC start time for rehydration in the format YYYY-MM-DDTHH:MM.
Ending Time	UTC end time for rehydration in the format YYYY-MM-DDTHH:MM.
Delete on Read	If true, objects are deleted after rehydration.
Enable Storage	Enable to specify a storage extension for tracking rehydration progress.
Storage Directory	Directory for storing rehydration state, useful for maintaining state and resuming operations. (Only relevant if Enable Storage is true)

Usage

Kubernetes

When deploying on Kubernetes, you must configuration authentication to AWS. This can be done by setting the following environment.

Edit the YAML manifest downloaded from Bindplane's agent install page. Add the following environment variables to the opentelemetry-container container.

spec:
  template:
    spec:
      containers:
        - name: opentelemetry-container
          env:
            - name: AWS_ACCESS_KEY_ID
              value: '<your access key>'
            - name: AWS_SECRET_ACCESS_KEY
              value: '<your secret key>'

Example Configuration

Basic Configuration

This configuration sets up AWS S3 Rehydration with necessary details such as Region, Bucket, and time range for rehydration.

Adjust the Region, Bucket, Starting Time, and Ending Time to match your configuration.