# Google Cloud Storage Pub/Sub Event

The Google Cloud Pub/Sub Event Receiver consumes GCS event notifications via a Pub/Sub subscription and emits the GCS object as the string body of a log record.

### Supported Platforms

| Platform | Supported |
| -------- | --------- |
| Linux    | ✓         |
| Windows  | ✓         |
| macOS    | ✓         |

Available in the Bindplane Distro for OpenTelemetry Collector `v1.95.0+`.

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* A Google Cloud project with access to Cloud Storage and Pub/Sub.
* A Pub/Sub subscription configured to receive GCS event notifications.
* Ensure the collector has permission to pull and acknowledge messages from the Pub/Sub subscription.
* Ensure the collector has permission to read objects from the GCS bucket.
* If using Application Default Credentials, ensure the environment is configured accordingly.

### How It Works <a href="#how-it-works" id="how-it-works"></a>

1. The receiver pulls messages from a Pub/Sub subscription for GCS event notifications.
2. When a notification is received, the receiver downloads the GCS object.
3. The receiver reads the object into the body of a new log record.
4. Messages undergo two-layer deduplication:
   * **Batch-level**: Within a single Pull response, duplicates keyed by (bucket, object, generation) are acknowledged immediately.
   * **Cross-batch**: A time-bounded tracker (configurable via `dedup_ttl`) catches sequential duplicates arriving in separate Pull responses, accounting for GCS's at-least-once delivery semantics.
5. If a GCS object is not found (404 error), the message is nacked for redelivery or DLQ processing.
6. If a permission denied error (403) is encountered, the message is nacked for redelivery or DLQ processing.
7. Large objects are handled via offset-based resumption when Redis offset storage is enabled, allowing arbitrarily large files to be processed without memory exhaustion.

### Configuration Fields <a href="#configuration-fields" id="configuration-fields"></a>

<table><thead><tr><th>Field</th><th width="111.65625">Type</th><th width="97.92578125">Default</th><th width="102.31640625">Required</th><th>Description</th></tr></thead><tbody><tr><td>project_id</td><td>string</td><td></td><td>true</td><td>The Google Cloud project ID that contains the Pub/Sub subscription.</td></tr><tr><td>subscription_id</td><td>string</td><td></td><td>true</td><td>The Pub/Sub subscription ID that receives GCS event notifications.</td></tr><tr><td>workers</td><td>int</td><td>5</td><td>false</td><td>The number of concurrent workers to process events.</td></tr><tr><td>max_extension</td><td>duration</td><td>3600</td><td>false</td><td>The maximum total time (in seconds) for which the receiver will extend the ack deadline for a message being processed. After this duration, the message becomes eligible for redelivery.</td></tr><tr><td>max_log_size</td><td>int</td><td>1048576</td><td>false</td><td>The maximum size in bytes for a single log record. Logs exceeding this size will be split into chunks.</td></tr><tr><td>max_logs_emitted</td><td>int</td><td>1000</td><td>false</td><td>The maximum number of log records to emit in a batch. A higher number will result in fewer batches, but more memory.</td></tr><tr><td>bucket_name_filter</td><td>string</td><td></td><td>false</td><td>When set, the source will only emit logs for bucket names that match the specified regex.</td></tr><tr><td>object_key_filter</td><td>string</td><td></td><td>false</td><td>When set, the source will only emit logs for object names that match the specified regex.</td></tr><tr><td>enable_offset_storage</td><td>bool</td><td>false</td><td>false</td><td>When enabled, the current position into an object will be saved to Redis, and reading will resume from where it left off after a collector restart.</td></tr><tr><td>redis_hostname</td><td>string</td><td></td><td>false</td><td>The hostname or IP address of the Redis server used for offset storage.</td></tr><tr><td>redis_port</td><td>int</td><td>6379</td><td>false</td><td>The port number of the Redis server used for offset storage.</td></tr><tr><td>redis_password</td><td>string</td><td></td><td>false</td><td>The password for the Redis server used for offset storage.</td></tr><tr><td>redis_database</td><td>int</td><td>0</td><td>false</td><td>The Redis database number to use for offset storage.</td></tr><tr><td>redis_expiration</td><td>int</td><td>0</td><td>false</td><td>The expiration time (in seconds) for offset storage in Redis.</td></tr><tr><td>redis_tls_enabled</td><td>bool</td><td>false</td><td>false</td><td>Whether to enable TLS for the Redis connection.</td></tr><tr><td>redis_tls_ca_file</td><td>string</td><td></td><td>false</td><td>The path to the CA file for Redis TLS connection.</td></tr><tr><td>redis_tls_cert_file</td><td>string</td><td></td><td>false</td><td>The path to the client certificate file for Redis TLS connection.</td></tr><tr><td>redis_tls_key_file</td><td>string</td><td></td><td>false</td><td>The path to the client key file for Redis TLS connection.</td></tr></tbody></table>

#### Example Configuration <a href="#example-configuration" id="example-configuration"></a>

<figure><img src="https://1405008107-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FgmiOMzBfoNFwmKJFHMcJ%2Fuploads%2FpyNwBCxwZgpDqUVgPx9K%2FScreenshot%202026-03-13%20at%208.23.55%E2%80%AFPM.png?alt=media&#x26;token=f3a75daf-35ca-44c0-9cab-79852376d253" alt=""><figcaption></figcaption></figure>

#### Component Telemetry <a href="#component-telemetry" id="component-telemetry"></a>

This component emits telemetry that can provide insight into how it is performing. The collector is configured to emit these metrics to `localhost:8888/metrics` by default.

<table><thead><tr><th>Metric Name</th><th width="162.04296875">Type</th><th>Description</th></tr></thead><tbody><tr><td>otelcol_gcsevent.batch_size</td><td>histogram</td><td>The number of logs in a batch.</td></tr><tr><td>otelcol_gcsevent.objects_handled</td><td>counter</td><td>The number of GCS objects processed by the receiver.</td></tr><tr><td>otelcol_gcsevent.failures</td><td>counter</td><td>The number of failures encountered while processing GCS objects.</td></tr><tr><td>otelcol_gcsevent.parse_errors</td><td>counter</td><td>The number of individual log records skipped due to parse errors within a GCS object.</td></tr><tr><td>otelcol_gcsevent.dlq_file_not_found_errors</td><td>counter</td><td>The number of file not found errors that triggered DLQ processing.</td></tr><tr><td>otelcol_gcsevent.dlq_iam_errors</td><td>counter</td><td>The number of IAM permission denied errors that triggered DLQ processing.</td></tr><tr><td>otelcol_gcsevent.dlq_unsupported_file_errors</td><td>counter</td><td>The number of unsupported file type errors that triggered DLQ processing.</td></tr></tbody></table>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.bindplane.com/integrations/sources/google-cloud-storage-pub-sub-event.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.