Chronosphere

Send your logs, metrics, and traces to Chronosphere using the OpenTelemetry Protocol (OTLP).

Supported Telemetry Types

Logs
Metrics
Traces

✓

✓

✓

Prerequisites

Before configuring the Chronosphere destination, ensure you have:

  • A Chronosphere account with tenant access

  • Your tenant name (e.g., mycompany from mycompany.chronosphere.io)

  • A service account API token with write permissions for your telemetry types

For help creating a service account and API token, see the Chronosphere service accounts documentation.

Configuration

Basic Configuration

The minimum configuration requires your tenant name and API token.

Parameter
Type
Default
Description

Choose Telemetry Type

telemetrySelector

["Logs","Metrics","Traces"]

Select which telemetry types to export (Logs, Metrics, Traces).

Tenant Name

string

""

Your Chronosphere tenant subdomain (e.g., mycompany).

API Token

string

""

Service account API token with write permissions. Sensitive field.

Protocol

enum

http

OTLP protocol: http (recommended) or grpc. For gRPC, collector connects to <tenant>.chronosphere.io:443 over TLS.

Advanced Configuration

TLS Configuration

TLS is enabled by default and required for gRPC protocol.

Parameter
Type
Default
Description

Enable TLS

bool

true

Enable TLS for secure connections (required for gRPC).

Skip TLS Certificate Verification

bool

false

Skip certificate validation (not recommended in production).

TLS Certificate Authority File

string

""

Optional CA certificate file for server validation.

Mutual TLS

bool

false

Enable client TLS authentication.

TLS Client Certificate File

string

""

Client certificate file for mTLS.

TLS Client Private Key

string

""

Client private key file for mTLS.

HTTP Headers and Compression

Parameter
Type
Default
Description

Additional Headers

map

{}

Custom HTTP headers for requests (HTTP protocol only).

Compression

enum

zstd

Compression algorithm for HTTP OTLP: none, gzip, or zstd.

Batching Configuration

Chronosphere recommends specific batch settings to prevent payload size violations.

Parameter
Type
Default
Description

Batch Send Size

int

1000

Number of items to trigger a batch send.

Batch Max Size

int

2000

Maximum items per batch before splitting.

Batch Timeout

int

1

Seconds before sending incomplete batch.

Request Timeout

int

30

Seconds to wait for request completion.

Queuing and Retry

Parameter
Type
Default
Description

Enable Retry on Failure

bool

true

Retry failed requests with exponential backoff.

Initial Interval

int

5

Seconds to wait after first failure.

Max Interval

int

30

Maximum backoff interval between retries.

Max Elapsed Time

int

300

Maximum total retry time in seconds.

Enable Sending Queue

bool

true

Buffer telemetry in memory before sending.

Number of Consumers

int

10

Concurrent batch processors from queue.

Queue Size

int

5000

Maximum batches in memory queue.

Enable Persistent Queuing

bool

true

Buffer telemetry to disk during outages.

Persistent Queue Directory

string

${OIQ_OTEL_COLLECTOR_HOME}/storage

Directory for disk-based queue.

Synchronize Persistent Queue to Disk

bool

false

fsync queue writes (safer but slower).

Telemetry-Specific Settings

Parameter
Type
Default
Description

Metrics Validation Response Verbosity

enum

SHORT

Error detail level: SHORT, SUMMARY, or DETAILED.

Convert Resource Attributes to Telemetry Attributes

bool

false

Map resource attributes to telemetry attributes.

Examples

gRPC Configuration with mTLS

Use gRPC with mutual TLS for secure, high-performance connections in zero-trust or highly regulated environments. This configuration enables client certificate authentication.

High-Volume Scenario

For environments sending large volumes of telemetry, increase buffering and processing capacity by configuring larger queue sizes, more concurrent consumers, and persistent queuing for reliability.

Configuration Tips

Protocol Selection and Compression

  • HTTP (recommended): Broadly compatible, works through most firewalls and proxies. Use zstd compression for best balance.

  • gRPC: Lower overhead and native binary protocol. Better for high-throughput scenarios but requires direct connectivity to port 443.

  • Compression: Use zstd for most cases; switch to gzip or none only if CPU usage is a concern.

Queue Configuration for Reliability

  • Enable Persistent Queuing for mission-critical data paths to survive restarts and longer outages.

  • Queue Size: Start with 5000; increase to 10,000 for high-volume burst scenarios.

  • Number of Consumers: Use 10-20 for high-volume environments; adjust based on CPU availability.

Timeout and Batch Settings

  • Batch Size: Keep at 1000 items (Chronosphere recommendation) for optimal throughput.

  • Timeout: Use 30 seconds default; increase to 45-60 seconds for high-latency or intercontinental networks.

  • Batch Timeout: Set to 1 second for real-time responsiveness with reasonable batching efficiency.

Troubleshooting

Authentication and Connection Issues

Symptoms: "401 Unauthorized", "Invalid API Token", or connection timeouts.

Solutions:

  1. Verify the API token is correct and has not expired in Chronosphere service accounts.

  2. Confirm the token has write permissions for the telemetry types you're sending.

  3. Test connectivity to <tenant>.chronosphere.io:443: curl -I https://<tenant>.chronosphere.io/otlp (for HTTP) or use telnet <tenant>.chronosphere.io 443 (for gRPC).

Metrics Rejection and Validation

Symptoms: "Partial success" responses or metrics rejected with validation errors.

Solutions:

  1. Set Metrics Validation Response Verbosity to DETAILED to see full error messages.

  2. Check for metrics exceeding Chronosphere limits: payload must be <16 MB compressed; metric items <10,000 per batch.

  3. Review Chronosphere's incoming data dashboard for cardinality or formatting issues.

Data Not Arriving

Symptoms: No data appears in Chronosphere despite successful configuration.

Solutions:

  1. Verify telemetry is being generated upstream (check source configuration logs).

  2. Confirm the correct telemetry types are enabled in both BindPlane and your sources.

  3. Check BindPlane and Chronosphere logs for validation errors or rate limiting.

Last updated

Was this helpful?