Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Bindplane Collector

The Bindplane Collector source collects the collector's own logs and internal telemetry from the collector the configuration is applied to. Logs are read from the collector's log files with the filelog receiver. Metrics are scraped from the collector's internal Prometheus endpoint and labeled with host resource attributes. Every metric the collector exposes is collected by default; individual metrics can be turned off with per-metric toggles. Use it to monitor the health of your fleet from within the pipelines they already run.

Migrating from the previous version

This is version 2 of the source. The previous version is deprecated but still functions.

  • Metrics are no longer restricted to the otelcol_* series. The previous version scraped the endpoint and kept only otelcol_* metrics. That hid every metric the collector emits from an instrumentation library, including http_server_request_duration_seconds, the receiver request-latency histogram. This version collects all metrics the collector exposes and lets you turn individual ones off.

  • The four metric checkboxes (Process, Receiver, Processor, Exporter) are replaced by a per-metric toggle control. Any metric left on is collected; any metric turned off is dropped by a filter processor. A newly added collector metric is collected until a toggle is added for it.

  • New filelog parameters are exposed: multiline parsing, the four file-name/path attribute toggles, encoding, and max log size. These let a Go panic and its stack trace arrive as one record instead of being split or truncated.

  • Defaults reproduce the previous version's metric set. A newly created v2 source collects the same otelcol_* metrics the v1 source did; the instrumentation-library metrics (HTTP, gRPC) start turned off. Turn them on as needed.

  • Metric names. By default the source keeps the raw Prometheus metric names, matching the previous version. Enable Trim Metric Suffixes to restore names closer to their OpenTelemetry form.

Supported Telemetry Types

Platform
Metrics
Logs
Traces

Linux

✓

✓

Windows

✓

✓

macOS

✓

✓

This is a Bindplane-internal source. It runs inside the Bindplane Collector and observes that same collector.

Prerequisites

This source is internal and needs no external system. It requires:

  • A Bindplane Collector running on the host. The source reads files written by that collector and scrapes its local metrics endpoint.

  • Read access to the collector log directory. With the defaults this is ${OIQ_OTEL_COLLECTOR_HOME}/log.

  • For metrics, the collector's internal telemetry endpoint must be reachable on the configured port (default 8888) on localhost. The collector exposes this Prometheus endpoint by default.

Configuration

Bindplane docs - Bindplane Collector - image 1

Telemetry

Parameter
Type
Required
Default
Description

Choose Telemetry Type

Telemetry Selector

No

Logs, Metrics

Which signals this source collects. Valid values are Logs and Metrics. The parameters shown below depend on the signals selected.

Logs

Shown when Logs is selected.

Parameter
Type
Required
Default
Description

Collector Log Path

String

Yes

${OIQ_OTEL_COLLECTOR_HOME}/log/collector.log

The absolute path to the Bindplane collector log.

Updater Log Path

String

Yes

${OIQ_OTEL_COLLECTOR_HOME}/log/updater.log

The absolute path to the Bindplane collector updater log.

Metrics

Shown when Metrics is selected.

Parameter
Type
Required
Default
Description

Metrics

Metrics

No

See below

Turn individual collector metrics on or off. Any metric left on is collected; any metric turned off is dropped. Any metric not listed is collected by default. See the Metric Reference for the full set.

Collector Metrics Port

Integer

Yes

8888

Port to scrape for collector metrics. Valid range is 0 to 65535.

Advanced

Parameter
Type
Required
Default
Description

Collect Standard Error Log

Boolean

No

true

Collect the collector's standard error log. This file is only present on Windows and macOS collectors. (Logs)

Standard Error Log Path (Windows)

String

Yes

${OIQ_OTEL_COLLECTOR_HOME}/log/observiq_collector.err

The absolute path to the collector standard error log on Windows. Shown when Collect Standard Error Log is enabled. (Logs)

Standard Error Log Path (macOS)

String

Yes

/var/log/observiq_collector.err

The absolute path to the collector standard error log on macOS. Shown when Collect Standard Error Log is enabled. (Logs)

Multiline Parsing

Enum: none, specify line start, specify line end

No

none

Enable multiline parsing so a multiline record, such as a Go panic and its stack trace, is read as one entry. (Logs)

Multiline Start Pattern

String

Yes

_(empty)_

Regex matching the start of a log entry. Shown when Multiline Parsing is set to specify line start. (Logs)

Multiline End Pattern

String

Yes

_(empty)_

Regex matching the end of a log entry. Shown when Multiline Parsing is set to specify line end. (Logs)

Include File Name Attribute

Boolean

No

true

Add the file name as the attribute log.file.name. (Logs)

Include File Path Attribute

Boolean

No

false

Add the file path as the attribute log.file.path. (Logs)

Include File Name Resolved Attribute

Boolean

No

false

Add the symlink-resolved file name as log.file.name_resolved. (Logs)

Include File Path Resolved Attribute

Boolean

No

false

Add the symlink-resolved file path as log.file.path_resolved. (Logs)

Encoding

Enum: nop, utf-8, utf-16le, utf-16be, ascii, big5

No

utf-8

The encoding of the log files being read. (Logs)

Max Log Size

String

Yes

1MiB

The maximum size of a single log entry. Larger entries are truncated. Raise this to keep large multiline records, such as Go panics, intact. (Logs)

Enable File Offset Storage

Boolean

No

true

When enabled, the current read position in each file is saved to disk so reading resumes where it left off after a collector restart. (Logs)

Offset Storage Directory

String

No

${OIQ_OTEL_COLLECTOR_HOME}/storage

Directory where the offset storage file is created. Shown when Enable File Offset Storage is enabled. (Logs)

Start At

Enum: beginning, end

No

end

Where in the log files to begin reading. (Logs)

Trim Metric Suffixes

Boolean

No

false

Trim unit and type suffixes (for example _total, _seconds, _bytes) from scraped metric names. Leave off to keep the raw Prometheus names, matching the deprecated source. (Metrics)

Collection Interval

Integer

No

60

How often, in seconds, to scrape for metrics. (Metrics)

Metric Reference

The Metrics control lists every metric the collector exposes at the supported collector version, grouped by category. A checked metric is collected; an unchecked metric is dropped by a filter processor. Metrics not listed here are collected by default. The otelcol_* categories are on by default (reproducing the deprecated source); the HTTP, gRPC, and Scrape Metadata categories, and the *_size metrics, are off by default.

The metric names below are the base Prometheus names. With Trim Metric Suffixes off (the default), the scraped names also carry the usual Prometheus suffixes, such as _total on counters and _seconds/_bytes on units.

Process Metrics

Metric
Default

otelcol_process_cpu_seconds

Enabled

otelcol_process_memory_rss

Enabled

otelcol_process_runtime_heap_alloc_bytes

Enabled

otelcol_process_runtime_total_alloc_bytes

Enabled

otelcol_process_runtime_total_sys_memory_bytes

Enabled

otelcol_process_uptime

Enabled

Receiver Metrics

Metric
Default

otelcol_receiver_accepted_log_records

Enabled

otelcol_receiver_accepted_metric_points

Enabled

otelcol_receiver_accepted_spans

Enabled

otelcol_receiver_accepted_profile_samples

Enabled

otelcol_receiver_refused_log_records

Enabled

otelcol_receiver_refused_metric_points

Enabled

otelcol_receiver_refused_spans

Enabled

otelcol_receiver_refused_profile_samples

Enabled

otelcol_receiver_failed_log_records

Enabled

otelcol_receiver_failed_metric_points

Enabled

otelcol_receiver_failed_spans

Enabled

otelcol_receiver_failed_profile_samples

Enabled

otelcol_receiver_requests

Enabled

otelcol_receiver_produced_items

Enabled

otelcol_receiver_produced_size1

Disabled

  1. The collector only emits *_size metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.

Processor Metrics

Metric
Default

otelcol_processor_incoming_items

Enabled

otelcol_processor_outgoing_items

Enabled

otelcol_processor_internal_duration

Enabled

otelcol_processor_consumed_items

Enabled

otelcol_processor_produced_items

Enabled

otelcol_processor_consumed_size1

Disabled

otelcol_processor_produced_size1

Disabled

  1. The collector only emits *_size metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.

Exporter Metrics

Metric
Default

otelcol_exporter_sent_log_records

Enabled

otelcol_exporter_sent_metric_points

Enabled

otelcol_exporter_sent_spans

Enabled

otelcol_exporter_sent_profile_samples

Enabled

otelcol_exporter_send_failed_log_records

Enabled

otelcol_exporter_send_failed_metric_points

Enabled

otelcol_exporter_send_failed_spans

Enabled

otelcol_exporter_send_failed_profile_samples

Enabled

otelcol_exporter_enqueue_failed_log_records

Enabled

otelcol_exporter_enqueue_failed_metric_points

Enabled

otelcol_exporter_enqueue_failed_spans

Enabled

otelcol_exporter_enqueue_failed_profile_samples

Enabled

otelcol_exporter_queue_size

Enabled

otelcol_exporter_queue_capacity

Enabled

otelcol_exporter_enqueue_size2

Enabled

otelcol_exporter_queue_batch_send_size2

Enabled

otelcol_exporter_in_flight_requests

Enabled

otelcol_exporter_consumed_items

Enabled

otelcol_exporter_consumed_size1

Disabled

  1. The collector only emits *_size metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.

  2. This toggle also covers its _bytes companion (otelcol_exporter_enqueue_size_bytes, otelcol_exporter_queue_batch_send_size_bytes).

Scraper Metrics

Metric
Default

otelcol_scraper_scraped_metric_points

Enabled

otelcol_scraper_scraped_log_records

Enabled

otelcol_scraper_errored_metric_points

Enabled

otelcol_scraper_errored_log_records

Enabled

Connector Metrics

Metric
Default

otelcol_connector_consumed_items

Enabled

otelcol_connector_produced_items

Enabled

otelcol_connector_consumed_size1

Disabled

otelcol_connector_produced_size1

Disabled

  1. The collector only emits *_size metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.

HTTP Server Metrics

Metric
Default

http_server_request_duration

Disabled

http_server_request_body_size

Disabled

http_server_response_body_size

Disabled

HTTP Client Metrics

Metric
Default

http_client_request_duration

Disabled

http_client_request_body_size

Disabled

gRPC Metrics

Metric
Default

rpc_server_call_duration

Disabled

rpc_client_call_duration

Disabled

Scrape Metadata Metrics

Emitted by the Prometheus receiver and metrics handler on every scrape, independent of what the endpoint serves. Off by default so a new v2 source matches the deprecated source.

Metric
Default

up

Disabled

scrape_duration_seconds

Disabled

scrape_samples_post_metric_relabeling

Disabled

scrape_samples_scraped

Disabled

scrape_series_added

Disabled

promhttp_metric_handler_errors_total

Disabled

Examples

Advanced log parameters

Keep a Go panic and its stack trace together as one record, add the file path attribute, and raise the max log size. The advanced log parameters are shown expanded.

Bindplane docs - Bindplane Collector - image 2

Turn on HTTP server request latency

The HTTP server metrics are off by default. Turn on http_server_request_duration to capture a receiver's request-latency histogram, for example to see a Splunk HEC source approaching its write timeout. In a resource manifest, the Metrics control stores the list of metrics that are turned off, so enabling a metric means removing it from that list.

Logs and metrics with stderr collection on Windows or macOS

The standard error log only exists on Windows and macOS collectors. The matching path is used automatically based on the platform the collector runs on.

Configuration Tips

  • The Metrics control is a denylist. Any metric you turn off is dropped by a filter processor. Any metric you leave on is collected, including a metric added in a newer collector that has no toggle yet. Curate the toggles as the collector's metric set grows.

  • HTTP and gRPC metrics are off by default so a new v2 source matches the deprecated source. Turn on http_server_request_duration when you need receiver request latency, such as watching a Splunk HEC source's write timeout.

  • The *_size metrics are off by default and also require the collector's own internal telemetry to be configured to emit them, so turning the toggle on is not sufficient on its own.

  • The standard error log is only written on Windows and macOS. On Linux the stderr paths are ignored even if Collect Standard Error Log is enabled.

  • Leave Enable File Offset Storage on for log collection so restarts do not re-read or skip records. The offset directory must be writable by the collector.

Troubleshooting

No collector metrics appear

Symptoms: the source produces logs but no metrics, or the metrics pipeline reports a scrape failure.

Solutions:

  1. Confirm Metrics is selected in Choose Telemetry Type.

  2. Verify the collector's internal telemetry endpoint is listening on the configured Collector Metrics Port (default 8888) on localhost. If the port was changed in the collector's telemetry settings, set Collector Metrics Port to match.

  3. Confirm the metrics you expect are not turned off in the Metrics control.

A specific metric is missing

Symptoms: most metrics appear, but a specific one does not.

Solutions:

  1. Check the Metrics control and confirm that metric is turned on.

  2. For a *_size metric, confirm the collector's own internal telemetry is configured to emit it. These are not emitted by default, so the toggle alone will not produce them.

  3. If you turned on Trim Metric Suffixes, remember that downstream metric names no longer carry the _total/_seconds/_bytes suffixes.

No collector logs appear

Symptoms: the source produces metrics but no logs, or the filelog receiver reports a missing file.

Solutions:

  1. Confirm Logs is selected and that Collector Log Path and Updater Log Path point at files that exist on the host. The defaults assume ${OIQ_OTEL_COLLECTOR_HOME} is set.

  2. If you expect older entries but see none, set Start At to beginning. With the default end, only new lines written after the source starts are read.

  3. Ensure the collector user can read the log directory and write the Offset Storage Directory.

A panic or large record is split or truncated

Symptoms: a Go panic and its stack trace arrive as several records, or a large record is cut off.

Solutions:

  1. Set Multiline Parsing to specify line start and provide a Multiline Start Pattern that matches the beginning of a log line, so the panic and its trace are read as one record.

  2. Raise Max Log Size above the default 1MiB if records are being truncated.

Standalone Source

Last updated

Was this helpful?