PricingSign inRequest a Demo

Chronosphere

Send your logs, metrics, and traces to Chronosphere using the OpenTelemetry Protocol (OTLP).

Supported Telemetry Types

Telemetry Type
Supported

Traces

✓

Metrics

✓

Logs

✓

Prerequisites

Before configuring the Chronosphere destination, you'll need:

  1. Active Chronosphere Account: A Chronosphere Observability Platform account with access to your organization's instance

  2. Tenant Name: Your organization's Chronosphere tenant name (the subdomain of your instance, e.g., if your URL is https://mycompany.chronosphere.io, your tenant name is mycompany)

  3. API Token: A valid API token from a Chronosphere service account with write permissions for the telemetry types you plan to send

Creating an API Token in Chronosphere

To generate an API token:

  1. Log in to your Chronosphere instance

  2. Navigate to Administration → Accounts & Teams → Service Accounts

  3. Click Create Service Account or select an existing account

  4. Create or view an API token with the following scopes:

    • Write metrics (for metrics data)

    • Write logs (for logs data)

    • Write traces (for traces data)

  5. Copy the generated token securely (you'll need this for the BindPlane configuration)

Configuration

Basic Connection Settings

To configure a basic Chronosphere OTLP destination:

  1. In BindPlane, create a new destination

  2. Select Chronosphere OTLP as the destination type

  3. Enter your configuration:

Parameter
Description
Required

Tenant Name

Your Chronosphere organization name (subdomain)

Yes

API Token

Service account API token with write permissions

Yes

Protocol

HTTP/1.1 or HTTP/2 protocol version

No (default: HTTP/1.1)

Compression

Compression algorithm: none, gzip, or zstd

No (default: zstd)

Telemetry Type Selection

Choose which telemetry types you want to send:

  • Logs: Enable to send structured log data to Chronosphere

  • Metrics: Enable to send metric data for monitoring

  • Traces: Enable to send distributed tracing data

You can enable any combination of these types.

Batching Configuration

Chronosphere recommends specific batch processor settings to prevent payload size violations:

Parameter
Recommended Value
Description

Batch Send Size

1000

Number of items to trigger batch sending

Batch Max Size

2000

Maximum items per batch before forcing split

Batch Timeout

1

Maximum time in seconds...

These defaults optimize data delivery while respecting Chronosphere's payload size limits:

  • Compressed payloads must be smaller than 16 MB

  • Metric payloads must contain fewer than 10,000 items

TLS & Security Configuration

Basic TLS (Certificate Verification)

For standard HTTPS connections with server certificate verification:

Parameter
Description

Enable TLS

Enable TLS encryption (required by Chronosphere)

CA Certificate File

Path to CA certificate file for server verification

Insecure Skip Verify

Skip server certificate verification (development only)

Mutual TLS (mTLS)

For mutual TLS authentication:

Parameter
Description

Enable TLS

Enable TLS encryption

CA Certificate File

Path to CA certificate for server verification

Client Certificate File

Path to client certificate file (mTLS)

Client Key File

Path to client private key file (mTLS)

Security Notes:

  • Never use Insecure Skip Verify in production

  • Store certificate files securely with appropriate permissions

  • Regularly rotate certificates according to your security policy

  • Consider using BindPlane's secret management for sensitive values

Advanced Configuration

HTTP Headers

Add custom HTTP headers to requests:

http_headers:
  X-Custom-Header: "value"
  Another-Header: "another-value"

Headers are merged with default headers (including the API-Token header).

Retry Configuration

Chronosphere recommends enabling retries to handle transient failures:

Parameter
Recommended Value
Description

Retry on Failure

Enabled

Automatically retry failed requests

Initial Interval

5

Maximum wait in seconds...

Max Interval

30

Maximum wait in seconds...

Max Elapsed Time

300

Maximum time in seconds...

Persistent Queuing

For high-volume scenarios or when reliability is critical:

Parameter
Description

Sending Queue Size

Number of items to buffer in memory

Persistent Queue Directory

Directory for disk-based storage (optional)

Persistent Queue Fsync

Force filesystem sync on each write (safety vs. performance)

When persistent queue is enabled, data is buffered to disk if the destination is unavailable, ensuring no data loss.

Metrics-Specific Options

For metrics telemetry:

Parameter
Options
Description

Metrics Validation Response

SHORT, SUMMARY, DETAILED

Verbosity of partial success responses

Resource to Telemetry Conversion

Enabled/Disabled

Convert resource attributes to metric attributes

Telemetry-Specific Processing

Logs Processing

When logs are enabled, the destination:

  • Collects structured log records

  • Removes the raw log copy attribute (log.record.original) to reduce payload size

  • Maintains all other log attributes and fields

Metrics Processing

When metrics are enabled:

  • All metric types (counters, gauges, histograms, exponential histograms) are supported

  • Resource attributes can be converted to metric attributes based on configuration

  • Partial success responses provide detailed validation feedback

  • Resource-to-telemetry conversion helps normalize data across different sources

Traces Processing

When traces are enabled:

  • Distributed trace spans are collected and exported

  • All span attributes and resource information is preserved

  • Trace context propagation is maintained for correlation

Configuration Examples

Minimal Configuration (Logs Only)

name: chronosphere-logs
type: chronosphere_otlp
parameters:
  tenant_name: mycompany
  api_token: ${CHRONOSPHERE_API_TOKEN}
  send_logs: true

Full Configuration (All Telemetry Types)

name: chronosphere-full
type: chronosphere_otlp
parameters:
  tenant_name: mycompany
  api_token: ${CHRONOSPHERE_API_TOKEN}
  send_logs: true
  send_metrics: true
  send_traces: true
  protocol: "http/1.1"
  compression: "zstd"
  batch_send_size: 1000
  batch_max_size: 2000
  batch_timeout: "1s"
  enable_tls: true
  retry_on_failure: true
  retry_initial_interval: "5s"
  retry_max_interval: "30s"
  retry_max_elapsed_time: "300s"

Mutual TLS Configuration

name: chronosphere-mtls
type: chronosphere_otlp
parameters:
  tenant_name: mycompany
  api_token: ${CHRONOSPHERE_API_TOKEN}
  send_logs: true
  send_metrics: true
  send_traces: true
  enable_tls: true
  ca_file: "/etc/ssl/certs/ca-bundle.crt"
  client_cert_file: "/etc/ssl/certs/client.crt"
  client_key_file: "/etc/ssl/private/client.key"
  mutual_tls: true
  retry_on_failure: true
  sending_queue_enabled: true
  sending_queue_num_consumers: 10
  sending_queue_queue_size: 5000
  persistent_queue_directory: "/var/lib/bindplane/queue"

High-Volume Optimization

For high-volume telemetry scenarios:

name: chronosphere-high-volume
type: chronosphere_otlp
parameters:
  tenant_name: mycompany
  api_token: ${CHRONOSPHERE_API_TOKEN}
  send_logs: true
  send_metrics: true
  send_traces: true
  protocol: "http/2"
  compression: "zstd"
  batch_send_size: 1500
  batch_max_size: 2000
  batch_timeout: "2s"
  timeout: "45s"
  sending_queue_enabled: true
  sending_queue_num_consumers: 20
  sending_queue_queue_size: 10000
  persistent_queue_directory: "/var/lib/bindplane/queue"
  persistent_queue_fsync: false
  retry_on_failure: true
  retry_initial_interval: "5s"
  retry_max_interval: "30s"
  retry_max_elapsed_time: "300s"

Performance Optimization

Batch Processing

The batch processor configuration significantly impacts performance:

  • Batch Send Size: Set to 1000 items (Chronosphere recommendation)

  • Batch Max Size: Set to 2000 items to prevent exceeding limits

  • Batch Timeout: Set to 1 second for real-time responsiveness with some batching efficiency

Larger batch sizes reduce overhead but increase latency; smaller sizes provide faster delivery but with more requests.

Compression

Chronosphere supports three compression algorithms:

Algorithm
Pros
Cons

zstd

Better compression, faster decompression (recommended)

Higher CPU usage for compression

gzip

Good compression, wide support

Slower decompression

none

No compression overhead

Larger payloads, more bandwidth

Use zstd for most scenarios; disable compression only for low-volume or development environments.

Queue Configuration

For reliability and burst handling:

  • Queue Size: Start with 5000-10000 for burst protection

  • Num Consumers: Use 10-20 concurrent senders for high volume

  • Persistent Directory: Enable for critical data paths

  • Fsync: Disable for better performance (data still safe with reasonable queue depth)

Timeout Settings

Chronosphere recommends:

  • Default Timeout: 30 seconds for most requests

  • High-Latency Networks: 45-60 seconds

  • HTTP/2: Can use same as HTTP/1.1

Troubleshooting

Authentication Errors

Error: "401 Unauthorized" or "Invalid API Token"

Solutions:

  1. Verify your API token is correct and hasn't expired

  2. Check that the service account has write permissions for your telemetry types

  3. Ensure the API token is properly formatted (no extra whitespace)

  4. Regenerate the token in Chronosphere if necessary

Connection Timeouts

Error: "Connection timeout" or "Request timeout"

Solutions:

  1. Check network connectivity to TENANT.chronosphere.io:443

  2. Increase the timeout setting from the default 30 seconds

  3. Verify firewall rules and proxies aren't blocking the connection

  4. Confirm your Chronosphere instance is operational

TLS/Certificate Errors

Error: "certificate verify failed" or "x509 certificate error"

Solutions:

  1. Verify the CA certificate file path is correct and readable

  2. Ensure the certificate is not expired

  3. For self-signed certificates, use the correct CA certificate file

  4. Check that the certificate CN or SAN matches the endpoint hostname

  5. For mutual TLS, verify both client certificate and key files are valid

Data Not Arriving

Error: No data appears in Chronosphere despite successful sending

Solutions:

  1. Verify data is being generated upstream (check source configuration)

  2. Test with a simple configuration first (logs only)

  3. Check Chronosphere's incoming data dashboard for validation errors

  4. Review the Chronosphere API token scopes for your telemetry type

  5. Verify the tenant name matches your Chronosphere instance

Partial Success Responses

Issue: Seeing "partial success" responses with rejected data

Troubleshooting:

  1. Check the Metrics Validation Response setting to understand rejection reasons

  2. Set to DETAILED for verbose error messages

  3. Review rejected metrics for cardinality or formatting issues

  4. Adjust cardinality reduction in upstream processors if needed

High CPU Usage

Issue: High CPU usage or slow performance

Solutions:

  1. Reduce batch size to decrease CPU time per batch

  2. Switch compression from zstd to gzip

  3. Increase batch timeout to allow more items per batch

  4. Disable persistent queue fsync if reliability margin allows

Related Resources

Support

For issues or questions:

  1. Check the troubleshooting section above

  2. Review your BindPlane logs for error details

  3. Verify Chronosphere status at https://chronosphere.io/

  4. Contact Chronosphere Support for platform-specific issues

  5. Contact BindPlane Support for integration issues

PreviousBindplane GatewayNextClickHouse

Last updated

Was this helpful?