# Running Collectors in Parallel on Linux and Windows

There are several scenarios where running more than one collector at the same time is useful: splitting a large host's telemetry sources across dedicated collector processes, forwarding data through a gateway, or validating a new collector version alongside the one currently in production. This guide covers the mechanics and common topologies.

### Running Multiple Collector Instances on the Same Host

The Bindplane Distro for OpenTelemetry (BDOT) is a standard binary process. You can run multiple instances on the same host as long as each instance uses a distinct:

* **Service name** — on Linux, each instance is a separate systemd unit.
* **Manager configuration file** — each instance needs its own `manager.yaml` pointing to its own data directory.
* **Listening ports** — any receiver port (OTLP gRPC, OTLP HTTP, Prometheus scrape, etc.) must not conflict with another instance on the same host.

#### Port Conflicts: Check Before Adding a Second Instance

Before installing a second collector, inventory the ports that the existing instance is already using.

**Linux:**

```bash
sudo ss -tlnp | grep observiq
```

Or by PID, if you know the process ID:

```bash
sudo ss -tlnp | grep <pid>
```

**Windows (PowerShell, run as Administrator):**

```powershell
Get-NetTCPConnection -State Listen | Where-Object { $_.OwningProcess -in (Get-Process observiq* | Select-Object -ExpandProperty Id) }
```

Or to list all listening ports with the owning process name:

```powershell
Get-NetTCPConnection -State Listen | Select-Object LocalPort, OwningProcess, @{Name='Process';Expression={(Get-Process -Id $_.OwningProcess -ErrorAction SilentlyContinue).Name}} | Sort-Object LocalPort
```

Common default ports used by BDOT receivers:

| Receiver               | Default Port | Protocol |
| ---------------------- | ------------ | -------- |
| OTLP gRPC              | 4317         | TCP      |
| OTLP HTTP              | 4318         | TCP      |
| Prometheus             | 8888         | TCP      |
| Health Check extension | 13133        | TCP      |

Assign the second instance different port numbers in its Bindplane configuration before deploying it.

#### Linux: Separate systemd Units

On Linux, each collector instance is managed as a separate systemd service. The package installer creates a unit named `observiq-otel-collector`. A second instance requires a distinct unit file and a separate configuration directory.

1. Copy the installed unit file and configuration directory to a new name:

```bash
sudo cp /usr/lib/systemd/system/observiq-otel-collector.service \
       /usr/lib/systemd/system/observiq-otel-collector-2.service

sudo cp -r /opt/observiq-otel-collector \
           /opt/observiq-otel-collector-2
```

2. Edit the new unit file to reference the second installation path:

```bash
sudo systemctl edit --force observiq-otel-collector-2
```

Update `ExecStart` and any path references to point to `/opt/observiq-otel-collector-2`.

3. Edit `/opt/observiq-otel-collector-2/manager.yaml` with the endpoint and secret key for the second managed collector registered in Bindplane.
4. Enable and start the second unit:

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now observiq-otel-collector-2
```

Each instance will appear as a separate agent in the Bindplane UI and can be assigned its own Configuration.

#### Windows: Separate Windows Services

On Windows, each collector instance runs as a separate Windows Service. The package installer creates a service named `observiq-otel-collector` with files in `C:\Program Files\observIQ OpenTelemetry Collector\`. A second instance requires its own installation directory and a uniquely named service.

1. Copy the existing installation directory to a new location (run PowerShell as Administrator):

```powershell
Copy-Item -Recurse "C:\Program Files\observIQ OpenTelemetry Collector" `
          "C:\Program Files\observIQ OpenTelemetry Collector 2"
```

2. Clear the `agent_id` in the new instance's `manager.yaml` so Bindplane registers it as a fresh agent, and update the `endpoint` and `secret_key` as needed:

```
C:\Program Files\observIQ OpenTelemetry Collector 2\manager.yaml
```

3. Create a new Windows Service pointing at the copied binary:

```powershell
New-Service -Name "observiq-otel-collector-2" `
            -BinaryPathName '"C:\Program Files\observIQ OpenTelemetry Collector 2\observiq-otel-collector.exe"' `
            -DisplayName "observIQ OpenTelemetry Collector 2" `
            -StartupType Automatic
```

4. Start the new service:

```powershell
Start-Service -Name "observiq-otel-collector-2"
```

5. Confirm the service is running and the agent has registered in Bindplane:

```powershell
Get-Service -Name "observiq-otel-collector-2"
```

{% hint style="warning" %}
Make sure the second instance's Bindplane Configuration assigns different receiver ports than the first instance uses. A port conflict will cause the second service to start but immediately fail to bind, logging an error rather than exiting the service cleanly.
{% endhint %}

To stop and remove the second service when it is no longer needed:

```powershell
Stop-Service -Name "observiq-otel-collector-2"
Remove-Service -Name "observiq-otel-collector-2"   # PowerShell 6+ / Windows Server 2019+
# On older systems use: sc.exe delete observiq-otel-collector-2
```

### Fan-Out Pattern: Multiple Independent Collectors

In the fan-out pattern, several independent collector instances each handle a distinct subset of telemetry sources. There is no inter-collector communication — each collector sends data directly to a destination.

```
Host A (Collector 1)  ──► Destination
Host B (Collector 2)  ──► Destination
Host C (Collector 3)  ──► Destination
```

**When to use it:**

* Separating security logs from infrastructure metrics on the same host to apply different processing pipelines.
* Isolating a high-volume source (for example, a busy application log) so that it cannot starve other sources of collector resources.
* Applying different destinations or different processing rules to different source categories without creating an overly complex single Configuration.

**How Bindplane helps:** Assign each collector a different Bindplane Configuration in the UI. Labels make this scalable — apply a label such as `role=app-logs` to a collector and then use a Label Selector rollout to push the correct Configuration to each group automatically.

### Gateway Fan-In Pattern: Multiple Collectors to a Single Gateway

In the gateway fan-in pattern, multiple collector instances forward telemetry to a single gateway collector. The gateway performs aggregation, enrichment, or routing before sending data to a final destination.

```
Collector 1 ──┐
Collector 2 ──┼──► Gateway Collector ──► Destination
Collector 3 ──┘
```

**When to use it:**

* Centralizing egress so that only the gateway needs credentials for the destination.
* Batching and compression at scale — collectors at the edge ship raw data; the gateway buffers and batches before forwarding.
* Enriching all telemetry with a common attribute set (for example, a datacenter name) in one place rather than in every edge collector's Configuration.

**Configuration approach in Bindplane:**

1. Create a Gateway Configuration that uses an OTLP receiver on a known host and port and forwards to your final destination.
2. Deploy that Configuration to the gateway host.
3. Create an Edge Configuration that uses an OTLP exporter pointing at the gateway's host and port (`host:4317` for gRPC).
4. Deploy that Configuration to the edge collector hosts.

{% hint style="info" %}
If the gateway and edge collectors are on separate hosts, make sure the gateway host's firewall allows inbound traffic on the OTLP ports from the edge collector hosts.
{% endhint %}

### Blue/Green Upgrades: Validate a New Collector Alongside the Old One

A blue/green upgrade lets you run a new collector version side-by-side with the current version, validate it, and then switch over — minimizing the risk of a bad upgrade affecting production telemetry.

**Process:**

1. **Install the new collector as a second instance** on the same host (or a parallel host), following the multi-instance steps above. Assign it a non-conflicting set of ports.
2. **Assign a test Configuration in Bindplane** that routes a copy of traffic — or a representative subset of sources — to a validation destination (for example, a staging index or a separate bucket).
3. **Validate.** Confirm the new collector is receiving and forwarding data as expected. Check:
   * The Bindplane UI shows the agent as connected and healthy.
   * Telemetry appears in the validation destination with the correct attributes and volume.
   * No errors appear in the new collector's logs.
4. **Switch over.** Reassign the original sources to the new collector's Configuration and stop the old instance.

Linux:

```bash
sudo systemctl stop observiq-otel-collector
sudo systemctl disable observiq-otel-collector
```

Windows (PowerShell, run as Administrator):

```powershell
Stop-Service -Name "observiq-otel-collector"
Set-Service -Name "observiq-otel-collector" -StartupType Disabled
```

5. **Promote the new instance.** Optionally rename the new unit back to the canonical name or leave it under its blue/green name for audit purposes.

{% hint style="success" %}
Use Bindplane's rollout feature to push the Configuration change to the new instance as a staged rollout. This lets you pause or roll back if problems surface during the validation window.
{% endhint %}

### Common Pitfalls

#### Port Conflicts

The most common failure when adding a second instance is a port conflict. If two collectors try to bind the same port, the second one will fail to start. Always check listening ports — `ss -tlnp` on Linux, `Get-NetTCPConnection -State Listen` on Windows — before configuring receivers for a new instance, and assign non-overlapping port numbers.

#### Duplicate Agent Registration

Each collector instance must have its own `agent_id` in `manager.yaml`. If you copy the `manager.yaml` from an existing instance and do not clear the `agent_id` field, the two instances may conflict in Bindplane. Let the new instance generate its own ID by removing or leaving blank the `agent_id` field before first startup.

#### Resource Contention

Multiple collector processes on the same host share CPU, memory, and disk I/O. Profile resource usage of the existing instance before adding more. A busy collector can easily use several hundred megabytes of RAM. On a constrained host, consider the fan-out pattern across separate hosts instead of co-locating multiple instances.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bindplane.com/how-to-guides/collector-management/running-collectors-in-parallel.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.