# Configuration

When operating a self-managed Prometheus instance, Bindplane's server configuration must be updated to connect to the remote Prometheus instance.

### Bindplane Configuration

After installing Bindplane, update the configuration file at `/etc/bindplane/config.yaml`using the editor of your choice.

* Set `prometheus.enableRemote` to `true`
* Set `prometheus.host` to the IP address or Hostname of your Prometheus server.

```yaml
prometheus:
  enableRemote: true
  localFolder: /var/lib/bindplane/prometheus
  host: prometheus.c.project.internal
  port: '9090'
  remoteWrite:
    endpoint: /api/v1/write
  auth:
    type: none
```

Once `enableRemote` and `host` are configured, restart the Bindplane server process.

```bash
sudo systemctl restart bindplane
```

At this point, Bindplane is installed and configured to use the remote Prometheus instance.

### Security

Prometheus supports several options for security. Basic authentication (Basic auth), Transport Layer\
Security (TLS), and Mutual TLS (mTLS).

#### Basic Authentication

Follow the Prometheus [Basic Auth Password Hashing](https://prometheus.io/docs/guides/basic-auth/#hashing-a-password) documentation to generate a password hash.

Once you have your hash, update `/etc/prometheus/web.yml` with your basic auth username and password hash.

// cspell:ignore maOicLymWgsIQleRCm604ePbaaavp9cKj3bJUg0IrcVXCHB3terLa

```yaml
# Example use only: admin:password
basic_auth_users:
  admin: $2b$12$maOicLymWgsIQleRCm604ePbaaavp9cKj3bJUg0IrcVXCHB3terLa
```

Restart the Prometheus service.

```bash
sudo systemctl restart prometheus
```

Test by making a curl request, without basic auth. You should expect a "401 Unauthorized" response.

```bash
curl -v -s localhost:9090/metrics > /dev/null
```

Test by making a curl request with your username and password.

```bash
curl -v -s -u 'admin:password' localhost:9090/metrics > /dev/null
```

You should expect a "200 OK" response. This will indicate that basic auth is working correctly.

Next, we need to update Bindplane with the new credentials. Edit `/etc/bindplane/config.yaml` on all of your Bindplane servers.

```yaml
prometheus:
  enableRemote: true
  localFolder: /var/lib/bindplane/prometheus
  host: prometheus.c.bpcli-dev.internal
  port: '9090'
  remoteWrite:
    endpoint: /api/v1/write
  auth:
    type: basic
    username: admin
    password: password
```

Restart the Bindplane service.

```bash
sudo systemctl restart bindplane
```

#### Transport Layer Security (TLS)

Copy the certificate keypair to `/etc/prometheus/tls`. The example commands assume that you have a certificate key pair in your working directory named `prometheus.crt` and `prometheus.key`

```bash
sudo mkdir /etc/prometheus/tls

sudo mv prometheus.crt prometheus.key /etc/prometheus/tls

sudo chown -R prometheus:prometheus /etc/prometheus/tls
sudo chmod 0600 \
  /etc/prometheus/tls/prometheus.crt \
  /etc/prometheus/tls/prometheus.key
```

Server side TLS can be configured by editing the web configuration file at `/etc/prometheus/web.yml` and configuring the certificate file and private key file paths.

```yaml
tls_server_config:
  cert_file: /etc/prometheus/tls/prometheus.crt
  key_file: /etc/prometheus/tls/prometheus.key
```

Restart the Prometheus service.

```bash
sudo systemctl restart prometheus
```

You can test if Prometheus is using TLS by using curl.

```bash
curl -kvs https://localhost:9090/metrics > /dev/null
```

You should expect a "200 OK" response. This will indicate that server side TLS is working correctly.

Next, we need to update Bindplane to use TLS when communicating with Prometheus. On all of your servers, perform the following steps.

Copy the certificate authority to `/etc/bindplane/tls`. The example commands assume that you have a certificate authority public key named `ca.crt` in your working directory.

```bash
sudo mkdir /etc/bindplane/tls
sudo mv ca.crt /etc/bindplane/tls

sudo chown -R bindplane:bindplane /etc/bindplane/tls
sudo chmod 0600 /etc/bindplane/tls/ca.crt
```

Edit `/etc/bindplane/config.yaml` on all of your Bindplane servers and add the `tls.tlsCa`\
parameter.

```yaml
prometheus:
  enableRemote: true
  localFolder: /var/lib/bindplane/prometheus
  host: prometheus.c.bpcli-dev.internal
  port: '9090'
  remoteWrite:
    endpoint: /api/v1/write
  auth:
    type: none
  enableTLS: true
  tls:
    tlsSkipVerify: false
    tlsCa:
      - /etc/bindplane/tls/ca.crt
```

{% hint style="info" %}
**NOTE**

Make sure `prometheus.host` matches the hostname of the Prometheus server's certificate. If the\
hostname does not match, you can set `prometheus.tls.tlsSkipVerify` to `true` to skip TLS verification. Skipping TLS verification is not recommended in a production environment.
{% endhint %}

Restart the Bindplane service.

```bash
sudo systemctl restart bindplane
```

#### Mutual TLS

Copy the certificate keypair and certificate authority to`/etc/prometheus/tls`. The example commands assume that you have a certificate key pair in your working directory named `prometheus.crt` and `prometheus.key` and a certificate authority named `ca.crt`.

```bash
sudo mkdir /etc/prometheus/tls

sudo mv prometheus.crt prometheus.key ca.crt /etc/prometheus/tls

sudo chown -R prometheus:prometheus /etc/prometheus/tls
sudo chmod 0600 \
  /etc/prometheus/tls/prometheus.crt \
  /etc/prometheus/tls/prometheus.key \
  /etc/prometheus/tls/ca.crt
```

Mutual TLS can be configured by editing the web configuration file at `/etc/prometheus/web.yml` and configuring the certificate file, private key file paths and certificate authority paths.

```yaml
tls_server_config:
  client_auth_type: RequireAndVerifyClientCert
  client_ca_file: /etc/prometheus/tls/ca.crt
  cert_file: /etc/prometheus/tls/prometheus.crt
  key_file: /etc/prometheus/tls/prometheus.key
```

Restart the Prometheus service.

```bash
sudo systemctl restart prometheus
```

You can test if Prometheus is using TLS by using curl on the Prometheus system.

```bash
# Sudo is required to read the TLS certificate files
# in /etc/prometheus/tls.
# Replace $(hostname -f) with the hostname that matches
# the prometheus server and certificate.
sudo curl -vs \
  --cacert /etc/prometheus/tls/ca.crt \
  --cert /etc/prometheus/tls/prometheus.crt \
  --key /etc/prometheus/tls/prometheus.key \
  "https://$(hostname -f):9090/metrics" > /dev/null
```

You should expect a "200 OK" response. This will indicate that mutual TLS is working correctly.

Next, we need to update Bindplane to use mutual TLS when communicating with Prometheus. On all of your servers, perform the following steps.

Copy the certificate authority and client keypair to `/etc/bindplane/tls`. The example commands assume that you have a certificate key pair in your working directory named `bindplane.crt` and `bindplane.key` and a certficate authority named `ca.crt`.

```bash
sudo mkdir /etc/bindplane/tls
sudo mv bindplane.crt bindplane.key ca.crt /etc/bindplane/tls

sudo chown -R bindplane:bindplane /etc/bindplane/tls
sudo chmod 0600 \
  /etc/bindplane/tls/bindplane.crt \
  /etc/bindplane/tls/bindplane.key \
  /etc/bindplane/tls/ca.crt
```

Edit `/etc/bindplane/config.yaml` on all of your Bindplane servers and add the `tls` parameters.

```yaml
prometheus:
  enableRemote: true
  localFolder: /var/lib/bindplane/prometheus
  host: prometheus.c.bpcli-dev.internal
  port: '9090'
  remoteWrite:
    endpoint: /api/v1/write
  auth:
    type: none
  enableTLS: true
  tls:
    tlsSkipVerify: false
    tlsCa:
      - /etc/bindplane/tls/ca.crt
    tlsCert: /etc/bindplane/tls/bindplane.crt
    tlsKey: /etc/bindplane/tls/bindplane.key
```

{% hint style="info" %}
**NOTE**

Make sure `prometheus.host` matches the hostname of the Prometheus server's certificate. If the\
hostname does not match, you can set `prometheus.tls.tlsSkipVerify` to `true` to skip TLS verification. Skipping TLS verification is not recommended in a production environment.
{% endhint %}

Restart the Bindplane service.

```bash
sudo systemctl restart bindplane
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bindplane.com/production-checklist/bindplane/high-availability/prometheus/configuration.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.