> For the complete documentation index, see [llms.txt](https://docs.bindplane.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bindplane.com/integrations/sources/bindplane-collector.md). # Bindplane Collector The Bindplane Collector source collects the collector's own logs and internal telemetry from the collector the configuration is applied to. Logs are read from the collector's log files with the filelog receiver. Metrics are scraped from the collector's internal Prometheus endpoint and labeled with host resource attributes. Every metric the collector exposes is collected by default; individual metrics can be turned off with per-metric toggles. Use it to monitor the health of your fleet from within the pipelines they already run. {% hint style="info" %} **Migrating from the previous version** This is version 2 of the source. The [previous version](/integrations/sources/bindplane-collector/bindplane-collector-deprecated.md) is deprecated but still functions. * **Metrics are no longer restricted to the `otelcol_*` series.** The previous version scraped the endpoint and kept only `otelcol_*` metrics. That hid every metric the collector emits from an instrumentation library, including `http_server_request_duration_seconds`, the receiver request-latency histogram. This version collects all metrics the collector exposes and lets you turn individual ones off. * **The four metric checkboxes (Process, Receiver, Processor, Exporter) are replaced by a per-metric toggle control.** Any metric left on is collected; any metric turned off is dropped by a filter processor. A newly added collector metric is collected until a toggle is added for it. * **New filelog parameters are exposed**: multiline parsing, the four file-name/path attribute toggles, encoding, and max log size. These let a Go panic and its stack trace arrive as one record instead of being split or truncated. * **Defaults reproduce the previous version's metric set.** A newly created v2 source collects the same `otelcol_*` metrics the v1 source did; the instrumentation-library metrics (HTTP, gRPC) start turned off. Turn them on as needed. * **Metric names.** By default the source keeps the raw Prometheus metric names, matching the previous version. Enable Trim Metric Suffixes to restore names closer to their OpenTelemetry form. {% endhint %} ### Supported Telemetry Types | Platform | Metrics | Logs | Traces | | -------- | ------- | ---- | ------ | | Linux | ✓ | ✓ | | | Windows | ✓ | ✓ | | | macOS | ✓ | ✓ | | This is a Bindplane-internal source. It runs inside the Bindplane Collector and observes that same collector. ### Prerequisites This source is internal and needs no external system. It requires: * A Bindplane Collector running on the host. The source reads files written by that collector and scrapes its local metrics endpoint. * Read access to the collector log directory. With the defaults this is `${OIQ_OTEL_COLLECTOR_HOME}/log`. * For metrics, the collector's internal telemetry endpoint must be reachable on the configured port (default `8888`) on `localhost`. The collector exposes this Prometheus endpoint by default. ### Configuration
Bindplane docs - Bindplane Collector - image 1
**Telemetry** | Parameter | Type | Required | Default | Description | | --------------------- | ------------------ | -------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------- | | Choose Telemetry Type | Telemetry Selector | No | Logs, Metrics | Which signals this source collects. Valid values are Logs and Metrics. The parameters shown below depend on the signals selected. | **Logs** Shown when Logs is selected. | Parameter | Type | Required | Default | Description | | ------------------ | ------ | -------- | ---------------------------------------------- | --------------------------------------------------------- | | Collector Log Path | String | Yes | `${OIQ_OTEL_COLLECTOR_HOME}/log/collector.log` | The absolute path to the Bindplane collector log. | | Updater Log Path | String | Yes | `${OIQ_OTEL_COLLECTOR_HOME}/log/updater.log` | The absolute path to the Bindplane collector updater log. | **Metrics** Shown when Metrics is selected. | Parameter | Type | Required | Default | Description | | ---------------------- | ------- | -------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Metrics | Metrics | No | See below | Turn individual collector metrics on or off. Any metric left on is collected; any metric turned off is dropped. Any metric not listed is collected by default. See the [Metric Reference](#metric-reference) for the full set. | | Collector Metrics Port | Integer | Yes | `8888` | Port to scrape for collector metrics. Valid range is 0 to 65535. | **Advanced** | Parameter | Type | Required | Default | Description | | ------------------------------------ | ------------------------------------------------------------- | -------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Collect Standard Error Log | Boolean | No | `true` | Collect the collector's standard error log. This file is only present on Windows and macOS collectors. (Logs) | | Standard Error Log Path (Windows) | String | Yes | `${OIQ_OTEL_COLLECTOR_HOME}/log/observiq_collector.err` | The absolute path to the collector standard error log on Windows. Shown when Collect Standard Error Log is enabled. (Logs) | | Standard Error Log Path (macOS) | String | Yes | `/var/log/observiq_collector.err` | The absolute path to the collector standard error log on macOS. Shown when Collect Standard Error Log is enabled. (Logs) | | Multiline Parsing | Enum: `none`, `specify line start`, `specify line end` | No | `none` | Enable multiline parsing so a multiline record, such as a Go panic and its stack trace, is read as one entry. (Logs) | | Multiline Start Pattern | String | Yes | `_(empty)_` | Regex matching the start of a log entry. Shown when Multiline Parsing is set to specify line start. (Logs) | | Multiline End Pattern | String | Yes | `_(empty)_` | Regex matching the end of a log entry. Shown when Multiline Parsing is set to specify line end. (Logs) | | Include File Name Attribute | Boolean | No | `true` | Add the file name as the attribute `log.file.name`. (Logs) | | Include File Path Attribute | Boolean | No | `false` | Add the file path as the attribute `log.file.path`. (Logs) | | Include File Name Resolved Attribute | Boolean | No | `false` | Add the symlink-resolved file name as `log.file.name_resolved`. (Logs) | | Include File Path Resolved Attribute | Boolean | No | `false` | Add the symlink-resolved file path as `log.file.path_resolved`. (Logs) | | Encoding | Enum: `nop`, `utf-8`, `utf-16le`, `utf-16be`, `ascii`, `big5` | No | `utf-8` | The encoding of the log files being read. (Logs) | | Max Log Size | String | Yes | `1MiB` | The maximum size of a single log entry. Larger entries are truncated. Raise this to keep large multiline records, such as Go panics, intact. (Logs) | | Enable File Offset Storage | Boolean | No | `true` | When enabled, the current read position in each file is saved to disk so reading resumes where it left off after a collector restart. (Logs) | | Offset Storage Directory | String | No | `${OIQ_OTEL_COLLECTOR_HOME}/storage` | Directory where the offset storage file is created. Shown when Enable File Offset Storage is enabled. (Logs) | | Start At | Enum: `beginning`, `end` | No | `end` | Where in the log files to begin reading. (Logs) | | Trim Metric Suffixes | Boolean | No | `false` | Trim unit and type suffixes (for example `_total`, `_seconds`, `_bytes`) from scraped metric names. Leave off to keep the raw Prometheus names, matching the deprecated source. (Metrics) | | Collection Interval | Integer | No | `60` | How often, in seconds, to scrape for metrics. (Metrics) | #### Metric Reference The Metrics control lists every metric the collector exposes at the supported collector version, grouped by category. A checked metric is collected; an unchecked metric is dropped by a filter processor. Metrics not listed here are collected by default. The `otelcol_*` categories are on by default (reproducing the deprecated source); the HTTP, gRPC, and Scrape Metadata categories, and the `*_size` metrics, are off by default. The metric names below are the base Prometheus names. With Trim Metric Suffixes off (the default), the scraped names also carry the usual Prometheus suffixes, such as `_total` on counters and `_seconds`/`_bytes` on units. **Process Metrics** | Metric | Default | | ------------------------------------------------ | ------- | | `otelcol_process_cpu_seconds` | Enabled | | `otelcol_process_memory_rss` | Enabled | | `otelcol_process_runtime_heap_alloc_bytes` | Enabled | | `otelcol_process_runtime_total_alloc_bytes` | Enabled | | `otelcol_process_runtime_total_sys_memory_bytes` | Enabled | | `otelcol_process_uptime` | Enabled | **Receiver Metrics** | Metric | Default | | -------------------------------------------- | -------- | | `otelcol_receiver_accepted_log_records` | Enabled | | `otelcol_receiver_accepted_metric_points` | Enabled | | `otelcol_receiver_accepted_spans` | Enabled | | `otelcol_receiver_accepted_profile_samples` | Enabled | | `otelcol_receiver_refused_log_records` | Enabled | | `otelcol_receiver_refused_metric_points` | Enabled | | `otelcol_receiver_refused_spans` | Enabled | | `otelcol_receiver_refused_profile_samples` | Enabled | | `otelcol_receiver_failed_log_records` | Enabled | | `otelcol_receiver_failed_metric_points` | Enabled | | `otelcol_receiver_failed_spans` | Enabled | | `otelcol_receiver_failed_profile_samples` | Enabled | | `otelcol_receiver_requests` | Enabled | | `otelcol_receiver_produced_items` | Enabled | | `otelcol_receiver_produced_size`1 | Disabled | 1. *The collector only emits `*_size` metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.* **Processor Metrics** | Metric | Default | | --------------------------------------------- | -------- | | `otelcol_processor_incoming_items` | Enabled | | `otelcol_processor_outgoing_items` | Enabled | | `otelcol_processor_internal_duration` | Enabled | | `otelcol_processor_consumed_items` | Enabled | | `otelcol_processor_produced_items` | Enabled | | `otelcol_processor_consumed_size`1 | Disabled | | `otelcol_processor_produced_size`1 | Disabled | 1. *The collector only emits `*_size` metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.* **Exporter Metrics** | Metric | Default | | ---------------------------------------------------- | -------- | | `otelcol_exporter_sent_log_records` | Enabled | | `otelcol_exporter_sent_metric_points` | Enabled | | `otelcol_exporter_sent_spans` | Enabled | | `otelcol_exporter_sent_profile_samples` | Enabled | | `otelcol_exporter_send_failed_log_records` | Enabled | | `otelcol_exporter_send_failed_metric_points` | Enabled | | `otelcol_exporter_send_failed_spans` | Enabled | | `otelcol_exporter_send_failed_profile_samples` | Enabled | | `otelcol_exporter_enqueue_failed_log_records` | Enabled | | `otelcol_exporter_enqueue_failed_metric_points` | Enabled | | `otelcol_exporter_enqueue_failed_spans` | Enabled | | `otelcol_exporter_enqueue_failed_profile_samples` | Enabled | | `otelcol_exporter_queue_size` | Enabled | | `otelcol_exporter_queue_capacity` | Enabled | | `otelcol_exporter_enqueue_size`2 | Enabled | | `otelcol_exporter_queue_batch_send_size`2 | Enabled | | `otelcol_exporter_in_flight_requests` | Enabled | | `otelcol_exporter_consumed_items` | Enabled | | `otelcol_exporter_consumed_size`1 | Disabled | 1. *The collector only emits `*_size` metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.* 2. *This toggle also covers its `_bytes` companion (`otelcol_exporter_enqueue_size_bytes`, `otelcol_exporter_queue_batch_send_size_bytes`).* **Scraper Metrics** | Metric | Default | | --------------------------------------- | ------- | | `otelcol_scraper_scraped_metric_points` | Enabled | | `otelcol_scraper_scraped_log_records` | Enabled | | `otelcol_scraper_errored_metric_points` | Enabled | | `otelcol_scraper_errored_log_records` | Enabled | **Connector Metrics** | Metric | Default | | --------------------------------------------- | -------- | | `otelcol_connector_consumed_items` | Enabled | | `otelcol_connector_produced_items` | Enabled | | `otelcol_connector_consumed_size`1 | Disabled | | `otelcol_connector_produced_size`1 | Disabled | 1. *The collector only emits `*_size` metrics when its own internal telemetry is configured to produce them, so enabling the toggle alone is not enough.* **HTTP Server Metrics** | Metric | Default | | -------------------------------- | -------- | | `http_server_request_duration` | Disabled | | `http_server_request_body_size` | Disabled | | `http_server_response_body_size` | Disabled | **HTTP Client Metrics** | Metric | Default | | ------------------------------- | -------- | | `http_client_request_duration` | Disabled | | `http_client_request_body_size` | Disabled | **gRPC Metrics** | Metric | Default | | -------------------------- | -------- | | `rpc_server_call_duration` | Disabled | | `rpc_client_call_duration` | Disabled | **Scrape Metadata Metrics** Emitted by the Prometheus receiver and metrics handler on every scrape, independent of what the endpoint serves. Off by default so a new v2 source matches the deprecated source. | Metric | Default | | --------------------------------------- | -------- | | `up` | Disabled | | `scrape_duration_seconds` | Disabled | | `scrape_samples_post_metric_relabeling` | Disabled | | `scrape_samples_scraped` | Disabled | | `scrape_series_added` | Disabled | | `promhttp_metric_handler_errors_total` | Disabled | ### Examples #### Advanced log parameters Keep a Go panic and its stack trace together as one record, add the file path attribute, and raise the max log size. The advanced log parameters are shown expanded.
Bindplane docs - Bindplane Collector - image 2
#### Turn on HTTP server request latency The HTTP server metrics are off by default. Turn on `http_server_request_duration` to capture a receiver's request-latency histogram, for example to see a Splunk HEC source approaching its write timeout. In a resource manifest, the Metrics control stores the list of metrics that are turned off, so enabling a metric means removing it from that list. ```yaml apiVersion: bindplane.observiq.com/v1 kind: Source metadata: name: bindplane-collector spec: type: bindplane-agent_v2 parameters: - name: telemetry_types value: - Metrics - name: metric_filtering value: - http_server_request_body_size - http_server_response_body_size - http_client_request_duration - http_client_request_body_size - rpc_server_call_duration - rpc_client_call_duration - otelcol_receiver_produced_size - otelcol_processor_consumed_size - otelcol_processor_produced_size - otelcol_exporter_consumed_size - otelcol_connector_consumed_size - otelcol_connector_produced_size ``` #### Logs and metrics with stderr collection on Windows or macOS The standard error log only exists on Windows and macOS collectors. The matching path is used automatically based on the platform the collector runs on. ```yaml apiVersion: bindplane.observiq.com/v1 kind: Source metadata: name: bindplane-collector spec: type: bindplane-agent_v2 parameters: - name: telemetry_types value: - Logs - Metrics - name: enable_stderr_log value: true - name: stderr_log_path_macos value: /var/log/observiq_collector.err - name: enable_offset_storage value: true - name: collection_interval value: 60 ``` ### Configuration Tips * The Metrics control is a denylist. Any metric you turn off is dropped by a filter processor. Any metric you leave on is collected, including a metric added in a newer collector that has no toggle yet. Curate the toggles as the collector's metric set grows. * HTTP and gRPC metrics are off by default so a new v2 source matches the deprecated source. Turn on `http_server_request_duration` when you need receiver request latency, such as watching a Splunk HEC source's write timeout. * The `*_size` metrics are off by default and also require the collector's own internal telemetry to be configured to emit them, so turning the toggle on is not sufficient on its own. * The standard error log is only written on Windows and macOS. On Linux the stderr paths are ignored even if Collect Standard Error Log is enabled. * Leave Enable File Offset Storage on for log collection so restarts do not re-read or skip records. The offset directory must be writable by the collector. ### Troubleshooting #### No collector metrics appear Symptoms: the source produces logs but no metrics, or the metrics pipeline reports a scrape failure. Solutions: 1. Confirm Metrics is selected in Choose Telemetry Type. 2. Verify the collector's internal telemetry endpoint is listening on the configured Collector Metrics Port (default `8888`) on `localhost`. If the port was changed in the collector's telemetry settings, set Collector Metrics Port to match. 3. Confirm the metrics you expect are not turned off in the Metrics control. #### A specific metric is missing Symptoms: most metrics appear, but a specific one does not. Solutions: 1. Check the Metrics control and confirm that metric is turned on. 2. For a `*_size` metric, confirm the collector's own internal telemetry is configured to emit it. These are not emitted by default, so the toggle alone will not produce them. 3. If you turned on Trim Metric Suffixes, remember that downstream metric names no longer carry the `_total`/`_seconds`/`_bytes` suffixes. #### No collector logs appear Symptoms: the source produces metrics but no logs, or the filelog receiver reports a missing file. Solutions: 1. Confirm Logs is selected and that Collector Log Path and Updater Log Path point at files that exist on the host. The defaults assume `${OIQ_OTEL_COLLECTOR_HOME}` is set. 2. If you expect older entries but see none, set Start At to `beginning`. With the default `end`, only new lines written after the source starts are read. 3. Ensure the collector user can read the log directory and write the Offset Storage Directory. #### A panic or large record is split or truncated Symptoms: a Go panic and its stack trace arrive as several records, or a large record is cut off. Solutions: 1. Set Multiline Parsing to specify line start and provide a Multiline Start Pattern that matches the beginning of a log line, so the panic and its trace are read as one record. 2. Raise Max Log Size above the default `1MiB` if records are being truncated. ### Standalone Source ```yaml apiVersion: bindplane.observiq.com/v1 kind: Source metadata: name: bindplane-collector spec: type: bindplane-agent_v2 parameters: - name: telemetry_types value: - Logs - Metrics - name: log_path value: log/collector.log - name: updater_log_path value: log/updater.log - name: enable_offset_storage value: true - name: offset_storage_dir value: storage - name: start_at value: end - name: metrics_port value: 8888 - name: collection_interval value: 60 - name: trim_metric_suffixes value: false ``` ### Related Resources * [Filelog Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver): reads the collector log files. * [Prometheus Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/prometheusreceiver): scrapes the collector's internal metrics endpoint. * [Collector internal telemetry](https://opentelemetry.io/docs/collector/internal-telemetry/): the metrics this source collects. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.bindplane.com/integrations/sources/bindplane-collector.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.