Running Collectors in Parallel on Linux and Windows

There are several scenarios where running more than one collector at the same time is useful: splitting a large host's telemetry sources across dedicated collector processes, forwarding data through a gateway, or validating a new collector version alongside the one currently in production. This guide covers the mechanics and common topologies.

Running Multiple Collector Instances on the Same Host

The Bindplane Distro for OpenTelemetry (BDOT) is a standard binary process. You can run multiple instances on the same host as long as each instance uses a distinct:

• Service name — on Linux, each instance is a separate systemd unit.

• Manager configuration file — each instance needs its own manager.yaml pointing to its own data directory.

• Listening ports — any receiver port (OTLP gRPC, OTLP HTTP, Prometheus scrape, etc.) must not conflict with another instance on the same host.

Port Conflicts: Check Before Adding a Second Instance

Before installing a second collector, inventory the ports that the existing instance is already using.

Linux:

sudo ss -tlnp | grep observiq

Or by PID, if you know the process ID:

sudo ss -tlnp | grep <pid>

Windows (PowerShell, run as Administrator):

Get-NetTCPConnection -State Listen | Where-Object { $_.OwningProcess -in (Get-Process observiq* | Select-Object -ExpandProperty Id) }

Or to list all listening ports with the owning process name:

Get-NetTCPConnection -State Listen | Select-Object LocalPort, OwningProcess, @{Name='Process';Expression={(Get-Process -Id $_.OwningProcess -ErrorAction SilentlyContinue).Name}} | Sort-Object LocalPort

Common default ports used by BDOT receivers:

Receiver	Default Port	Protocol
OTLP gRPC	4317	TCP
OTLP HTTP	4318	TCP
Prometheus	8888	TCP
Health Check extension	13133	TCP

Assign the second instance different port numbers in its Bindplane configuration before deploying it.

Linux: Separate systemd Units

On Linux, each collector instance is managed as a separate systemd service. The package installer creates a unit named observiq-otel-collector. A second instance requires a distinct unit file and a separate configuration directory.

1. Copy the installed unit file and configuration directory to a new name:

sudo cp /usr/lib/systemd/system/observiq-otel-collector.service \
       /usr/lib/systemd/system/observiq-otel-collector-2.service

sudo cp -r /opt/observiq-otel-collector \
           /opt/observiq-otel-collector-2

1. Edit the new unit file to reference the second installation path:

sudo systemctl edit --force observiq-otel-collector-2

Update ExecStart and any path references to point to /opt/observiq-otel-collector-2.

1. Edit /opt/observiq-otel-collector-2/manager.yaml with the endpoint and secret key for the second managed collector registered in Bindplane.

2. Enable and start the second unit:

sudo systemctl daemon-reload
sudo systemctl enable --now observiq-otel-collector-2

Each instance will appear as a separate agent in the Bindplane UI and can be assigned its own Configuration.

Windows: Separate Windows Services

On Windows, each collector instance runs as a separate Windows Service. The package installer creates a service named observiq-otel-collector with files in C:\Program Files\observIQ OpenTelemetry Collector\. A second instance requires its own installation directory and a uniquely named service.

1. Copy the existing installation directory to a new location (run PowerShell as Administrator):

Copy-Item -Recurse "C:\Program Files\observIQ OpenTelemetry Collector" `
          "C:\Program Files\observIQ OpenTelemetry Collector 2"

1. Clear the agent_id in the new instance's manager.yaml so Bindplane registers it as a fresh agent, and update the endpoint and secret_key as needed:

C:\Program Files\observIQ OpenTelemetry Collector 2\manager.yaml

1. Create a new Windows Service pointing at the copied binary:

New-Service -Name "observiq-otel-collector-2" `
            -BinaryPathName '"C:\Program Files\observIQ OpenTelemetry Collector 2\observiq-otel-collector.exe"' `
            -DisplayName "observIQ OpenTelemetry Collector 2" `
            -StartupType Automatic

1. Start the new service:

Start-Service -Name "observiq-otel-collector-2"

1. Confirm the service is running and the agent has registered in Bindplane:

Get-Service -Name "observiq-otel-collector-2"

Make sure the second instance's Bindplane Configuration assigns different receiver ports than the first instance uses. A port conflict will cause the second service to start but immediately fail to bind, logging an error rather than exiting the service cleanly.

To stop and remove the second service when it is no longer needed:

Stop-Service -Name "observiq-otel-collector-2"
Remove-Service -Name "observiq-otel-collector-2"   # PowerShell 6+ / Windows Server 2019+
# On older systems use: sc.exe delete observiq-otel-collector-2

Fan-Out Pattern: Multiple Independent Collectors

In the fan-out pattern, several independent collector instances each handle a distinct subset of telemetry sources. There is no inter-collector communication — each collector sends data directly to a destination.

Host A (Collector 1)  ──► Destination
Host B (Collector 2)  ──► Destination
Host C (Collector 3)  ──► Destination

When to use it:

• Separating security logs from infrastructure metrics on the same host to apply different processing pipelines.

• Isolating a high-volume source (for example, a busy application log) so that it cannot starve other sources of collector resources.

• Applying different destinations or different processing rules to different source categories without creating an overly complex single Configuration.

How Bindplane helps: Assign each collector a different Bindplane Configuration in the UI. Labels make this scalable — apply a label such as role=app-logs to a collector and then use a Label Selector rollout to push the correct Configuration to each group automatically.

Gateway Fan-In Pattern: Multiple Collectors to a Single Gateway

In the gateway fan-in pattern, multiple collector instances forward telemetry to a single gateway collector. The gateway performs aggregation, enrichment, or routing before sending data to a final destination.

Collector 1 ──┐
Collector 2 ──┼──► Gateway Collector ──► Destination
Collector 3 ──┘

When to use it:

• Centralizing egress so that only the gateway needs credentials for the destination.

• Batching and compression at scale — collectors at the edge ship raw data; the gateway buffers and batches before forwarding.

• Enriching all telemetry with a common attribute set (for example, a datacenter name) in one place rather than in every edge collector's Configuration.

Configuration approach in Bindplane:

1. Create a Gateway Configuration that uses an OTLP receiver on a known host and port and forwards to your final destination.

2. Deploy that Configuration to the gateway host.

3. Create an Edge Configuration that uses an OTLP exporter pointing at the gateway's host and port (host:4317 for gRPC).

4. Deploy that Configuration to the edge collector hosts.

If the gateway and edge collectors are on separate hosts, make sure the gateway host's firewall allows inbound traffic on the OTLP ports from the edge collector hosts.

Blue/Green Upgrades: Validate a New Collector Alongside the Old One

A blue/green upgrade lets you run a new collector version side-by-side with the current version, validate it, and then switch over — minimizing the risk of a bad upgrade affecting production telemetry.

Process:

1. Install the new collector as a second instance on the same host (or a parallel host), following the multi-instance steps above. Assign it a non-conflicting set of ports.

2. Assign a test Configuration in Bindplane that routes a copy of traffic — or a representative subset of sources — to a validation destination (for example, a staging index or a separate bucket).

3. Validate. Confirm the new collector is receiving and forwarding data as expected. Check:

   • The Bindplane UI shows the agent as connected and healthy.

   • Telemetry appears in the validation destination with the correct attributes and volume.

   • No errors appear in the new collector's logs.

4. Switch over. Reassign the original sources to the new collector's Configuration and stop the old instance.

Linux:

sudo systemctl stop observiq-otel-collector
sudo systemctl disable observiq-otel-collector

Windows (PowerShell, run as Administrator):

Stop-Service -Name "observiq-otel-collector"
Set-Service -Name "observiq-otel-collector" -StartupType Disabled

1. Promote the new instance. Optionally rename the new unit back to the canonical name or leave it under its blue/green name for audit purposes.

Use Bindplane's rollout feature to push the Configuration change to the new instance as a staged rollout. This lets you pause or roll back if problems surface during the validation window.

Common Pitfalls

Port Conflicts

The most common failure when adding a second instance is a port conflict. If two collectors try to bind the same port, the second one will fail to start. Always check listening ports — ss -tlnp on Linux, Get-NetTCPConnection -State Listen on Windows — before configuring receivers for a new instance, and assign non-overlapping port numbers.

Duplicate Agent Registration

Each collector instance must have its own agent_id in manager.yaml. If you copy the manager.yaml from an existing instance and do not clear the agent_id field, the two instances may conflict in Bindplane. Let the new instance generate its own ID by removing or leaving blank the agent_id field before first startup.

Resource Contention

Multiple collector processes on the same host share CPU, memory, and disk I/O. Profile resource usage of the existing instance before adding more. A busy collector can easily use several hundred megabytes of RAM. On a constrained host, consider the fan-out pattern across separate hosts instead of co-locating multiple instances.