# Switching Queues

## How to Prevent Data Loss When Switching Queues?

The principal danger when changing a destination's persistent queue configuration is data loss.

## Dangerous Edits

If a destination has a specific persistent queue configured, persists data to that queue, and then is restarted with a new persistent queue configuration before it is able to dequeue the persisted data, that data will be trapped in the old queue forever.

### Changing the persistent queue storage type

<figure><img src="/files/qOJftzCnN1GI9MXdsyba" alt=""><figcaption></figcaption></figure>

Changing from one persistent queue storage solution to another will result in a new queue being made for the new solution, with the old queue untouched.

### Changing the storage directory

<figure><img src="/files/suQhVQCQH4FYkNihctrC" alt=""><figcaption></figcaption></figure>

Changing the persistent queue directory will result in a new queue being stored in the new location, with the old queue untouched.

## Safe Edits

This describes the steps to modify a destination's persistent queue configuration safely without data loss. If data loss is not a significant concern, the queue configuration can be modified however desired.

When the persistent queue is empty for a destination, its settings can be safely modified directly in the destination. Queue state can be monitored through [Internal Telemetry](/configuration/bindplane-otel-collector/internal-telemetry.md).

When the persistent queue is not empty, safe queue modification requires the [Custom](https://docs.bindplane.com/integrations/sources/custom) source and a `Version 2` configuration.

<div><figure><img src="/files/Thm2HVCFfheJXuMnq4gd" alt=""><figcaption></figcaption></figure> <figure><img src="/files/fjHaddSHfShM5dSUEzog" alt=""><figcaption></figcaption></figure></div>

Let's say that you have a simple configuration sending TCP logs to an OTLP destination using a Bolt persistent queue.

<figure><img src="/files/DkNllDRib17VXkk6EYoH" alt=""><figcaption></figcaption></figure>

Next, let's say that:

* this configuration is rolled out to an agent
* the Bolt persistent queue is not empty
* we'd like to change the persistent queue to be a Pebble persistent queue.

In order to achieve this, we're going to have to alter our configuration to have two pipelines - one that drains the existing Bolt persistent queue, and one which reroutes any new data to our new Pebble persistent queue. This can be achieved with a few easy steps:

First, we're going to duplicate our existing OTLP destination, but tweak the duplicate to use the Pebble persistent queue instead. Add this duplicate as a second destination to the configuration.

<figure><img src="/files/OZ4OpHka7XY04Q9oax7o" alt=""><figcaption></figcaption></figure>

Second, we're going to add a new Custom source that won't send any logs.

<figure><img src="/files/b5fpz3J2arFKF6jlvNyL" alt=""><figcaption></figcaption></figure>

Third, we're going to reroute all existing traffic away from the Bolt destination, to the Pebble destination.

<figure><img src="/files/0aTqMdjMGL358ny46Blb" alt=""><figcaption></figcaption></figure>

Finally, we're going to add a route from the Custom receiver to the Bolt destination.

<figure><img src="/files/SfIoNgkCBCb2YuqYvc0R" alt=""><figcaption></figcaption></figure>

Once this is done, the configuration can be rolled out to the agent. The top pipeline will ensure that any new TCP Logs received are routed to the OTLP destination using Pebble as a persistent queue storage solution. The bottom pipeline will ensure that no new data will be sent to the Bolt persistent queue, and the existing data in the Bolt persistent queue will be sent to the OTLP destination.

Once the Bolt persistent queue has been drained (again, this can be monitored via [Internal Telemetry](/configuration/bindplane-otel-collector/internal-telemetry.md)), the bottom pipeline can be removed from the configuration, and it can be rolled out for a final time to the agent.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bindplane.com/configuration/bindplane-otel-collector/persistent-queue/switching-queues.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
