# Google Cloud Pub/Sub (Pull)

### Description

Read messages from a Google Cloud Pub/Sub topic and generate logs, metrics, or traces from the result.

{% hint style="warning" %}
**IMPORTANT**

The OpenTelemetry receiver this source uses is in beta and is subject to change.
{% endhint %}

### Supported Platforms

| Platform | Supported |
| -------- | --------- |
| Linux    | ✓         |
| Windows  | ✓         |
| macOS    | ✓         |

#### Configuration

<table><thead><tr><th width="161.69140625">Field</th><th>Description</th></tr></thead><tbody><tr><td>Telemetry Types</td><td>The types of telemetry to read from Pub/Sub.</td></tr><tr><td>Project ID</td><td>The Google Cloud project ID where the Pub/Sub client will connect to.</td></tr><tr><td>Subscription Name</td><td>Name of the pre-existing Pub/Sub subscription to read messages from.</td></tr><tr><td>Payload Encoding</td><td>Use "none" for attribute based detection. If the "content-type" attribute is missing from data, this is used to determine the encoding. Metrics and traces can only be OTLP Proto. See <a href="google-cloud-pubsub">Attribute Labels</a> for more information.</td></tr><tr><td>Payload Compression</td><td>Use "none" for attribute based detection. If the "content-encoding" attribute is missing from data, this is used to determine the compression. See <a href="google-cloud-pubsub">Attribute Labels</a> for more information.</td></tr><tr><td>Client ID</td><td>The client ID to be used by Pub/Sub to make load balancing decisions.</td></tr><tr><td>User Agent</td><td>The user agent that will be used by the Pub/Sub client to connect to the service.</td></tr><tr><td>Timeout</td><td>Timeout for all API calls, default is 12 seconds if unset.</td></tr><tr><td>Endpoint</td><td>Override of the default Pub/Sub endpoint. Leave empty for default.</td></tr><tr><td>Insecure</td><td>If using a custom endpoint, determines if connection should be insecure.</td></tr></tbody></table>

**Authentication**

The service account used by the collector should have the `Pub/Sub Subscriber` permission to read from the subscription. Additionally, make sure the subscription channel being read from is configured with `Pull` delivery.

Authentication is handled via the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. This should point to a file containing the service account key for the project.

To set this for the collector, you need to modify the service file depending on your OS.

**Linux**

Use a [Systemd override](https://wiki.archlinux.org/title/systemd):

```bash
sudo systemctl edit observiq-otel-collector
```

Then run the following command to reload the systemd configuration:

```bash
sudo systemctl daemon-reload
sudo systemctl restart observiq-otel-collector
```

**Windows**

Start powershell as administrator and run the following command:

```powershell
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\observiq-otel-collector" /v Environment /t REG_MULTI_SZ /d "GOOGLE_APPLICATION_CREDENTIALS=<path to key file>" /f
```

Then restart the collector service:

```powershell
Restart-Service observiq-otel-collector
```

**MacOS**

Add `GOOGLE_APPLICATION_CREDENTIALS` to the `EnvironmentVariables` dict in the launchd service file `/Library/LaunchDaemons/com.observiq.collector.plist` (other values are shown for context):

```xml
<key>EnvironmentVariables</key>
<dict>
    <key>GOOGLE_APPLICATION_CREDENTIALS</key>
    <string><path to key file></string>
    <key>OIQ_OTEL_COLLECTOR_HOME</key>
    <string>/opt/observiq-otel-collector/</string>
    <key>OIQ_OTEL_COLLECTOR_STORAGE</key>
    <string>/opt/observiq-otel-collector/storage</string>
</dict>
```

Then restart the collector:

```bash
sudo launchctl unload /Library/LaunchDaemons/com.observiq.collector.plist
sudo launchctl load /Library/LaunchDaemons/com.observiq.collector.plist
```

**Attribute Labels**

The Google Cloud PubSub receiver looks for the following attributes on the message to determine the message encoding and compression:

* `content-type` - How the message content should be interpreted.
  * If this value is set to `application/protobuf`, another attribute, `ce-type`, is expected to determine telemetry type. This attribute should be one of `org.opentelemetry.otlp.traces.v1`, `org.opentelemetry.otlp.metrics.v1`, or `org.opentelemetry.otlp.logs.v1`.
* `content-encoding` - Despite being called the "content-encoding" this controls the compression of the message.

If these attributes are present on the message data, the `Payload Encoding` and `Payload Compression` source parameters are ignored and should be set to "none".

If these attributes are not present on the message data, the receiver will use the `Payload Encoding` and `Payload Compression` parameters to determine the encoding and compression.

Metrics and traces can only be OTLP Proto. Logs can be OTLP JSON, OTLP Proto, or Raw Text. For more information, see the [OpenTelemetry Receiver Documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/googlecloudpubsubreceiver#encoding).

### Example Configuration

In this example, we are parsing log data from the given PubSub subscription. The expected [attributes](https://docs.bindplane.com/integrations/sources/google-cloud-pubsub) are not present on the PubSub message data, so the `Payload Encoding` , `Text Encoding` , and `Text Unmarshaling Separator` are used to determine the encoding, and `Payload Compression` is used to determine compression.

If these attributes were present, then the `Payload Encoding` and `Payload Compression` parameters should be set to "none".

**Web Interface**

<figure><img src="https://1405008107-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FgmiOMzBfoNFwmKJFHMcJ%2Fuploads%2F0t1TOQGuvIiRAsK49yKi%2Fimage.png?alt=media&#x26;token=3b1c5641-89f2-4616-8711-444278de1734" alt=""><figcaption></figcaption></figure>

**Standalone Source**

```yaml
apiVersion: bindplane.observiq.com/v1
kind: Source
metadata:
    name: pubsubpull
spec:
    type: googlecloudpubsub_v2
    parameters:
        - name: telemetry_types
          value:
            - Logs
            - Metrics
            - Traces
        - name: project
          value: project-id
        - name: subscription
          value: sub-name
        - name: compression
          value: none
        - name: client_id
          value: ""
        - name: user_agent
          value: ""
        - name: timeout
          value: 12s
        - name: endpoint
          value: ""
        - name: insecure
          value: false
        - name: encoding_extension
          value:
            disabled: false
            parameters:
                - name: text_encoding
                  value: utf8
                - name: text_unmarshaling_separator
                  value: \n?\r
            type: text_encoding_unmarshaling
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bindplane.com/integrations/sources/google-cloud-pubsub.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.