# Google Cloud Storage Pub/Sub Event

The Google Cloud Pub/Sub Event Receiver consumes GCS event notifications via a Pub/Sub subscription and emits the GCS object as the string body of a log record.

### Supported Platforms

| Platform | Supported |
| -------- | --------- |
| Linux    | ✓         |
| Windows  | ✓         |
| macOS    | ✓         |

Available in the Bindplane Distro for OpenTelemetry Collector `v1.95.0+`.

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* A Google Cloud project with access to Cloud Storage and Pub/Sub.
* A Pub/Sub subscription configured to receive GCS event notifications.
* Ensure the collector has permission to pull and acknowledge messages from the Pub/Sub subscription.
* Ensure the collector has permission to read objects from the GCS bucket.
* If using Application Default Credentials, ensure the environment is configured accordingly.

### How It Works <a href="#how-it-works" id="how-it-works"></a>

1. The receiver pulls messages from a Pub/Sub subscription for GCS event notifications.
2. When a notification is received, the receiver downloads the GCS object.
3. The receiver reads the object into the body of a new log record.
4. Messages undergo two-layer deduplication:
   * **Batch-level**: Within a single Pull response, duplicates keyed by (bucket, object, generation) are acknowledged immediately.
   * **Cross-batch**: A time-bounded tracker (configurable via `dedup_ttl`) catches sequential duplicates arriving in separate Pull responses, accounting for GCS's at-least-once delivery semantics.
5. If a GCS object is not found (404 error), the message is nacked for redelivery or DLQ processing.
6. If a permission denied error (403) is encountered, the message is nacked for redelivery or DLQ processing.
7. Large objects are handled via offset-based resumption when Redis offset storage is enabled, allowing arbitrarily large files to be processed without memory exhaustion.

### Configuration Fields <a href="#configuration-fields" id="configuration-fields"></a>

<table><thead><tr><th>Field</th><th width="111.65625">Type</th><th width="97.92578125">Default</th><th width="102.31640625">Required</th><th>Description</th></tr></thead><tbody><tr><td>project_id</td><td>string</td><td></td><td>true</td><td>The Google Cloud project ID that contains the Pub/Sub subscription.</td></tr><tr><td>subscription_id</td><td>string</td><td></td><td>true</td><td>The Pub/Sub subscription ID that receives GCS event notifications.</td></tr><tr><td>workers</td><td>int</td><td>5</td><td>false</td><td>The number of concurrent workers to process events.</td></tr><tr><td>max_extension</td><td>duration</td><td>3600</td><td>false</td><td>The maximum total time (in seconds) for which the receiver will extend the ack deadline for a message being processed. After this duration, the message becomes eligible for redelivery.</td></tr><tr><td>max_log_size</td><td>int</td><td>1048576</td><td>false</td><td>The maximum size in bytes for a single log record. Logs exceeding this size will be split into chunks.</td></tr><tr><td>max_logs_emitted</td><td>int</td><td>1000</td><td>false</td><td>The maximum number of log records to emit in a batch. A higher number will result in fewer batches, but more memory.</td></tr><tr><td>bucket_name_filter</td><td>string</td><td></td><td>false</td><td>When set, the source will only emit logs for bucket names that match the specified regex.</td></tr><tr><td>object_key_filter</td><td>string</td><td></td><td>false</td><td>When set, the source will only emit logs for object names that match the specified regex.</td></tr><tr><td>enable_offset_storage</td><td>bool</td><td>false</td><td>false</td><td>When enabled, the current position into an object will be saved to Redis, and reading will resume from where it left off after a collector restart.</td></tr><tr><td>redis_hostname</td><td>string</td><td></td><td>false</td><td>The hostname or IP address of the Redis server used for offset storage.</td></tr><tr><td>redis_port</td><td>int</td><td>6379</td><td>false</td><td>The port number of the Redis server used for offset storage.</td></tr><tr><td>redis_password</td><td>string</td><td></td><td>false</td><td>The password for the Redis server used for offset storage.</td></tr><tr><td>redis_database</td><td>int</td><td>0</td><td>false</td><td>The Redis database number to use for offset storage.</td></tr><tr><td>redis_expiration</td><td>int</td><td>0</td><td>false</td><td>The expiration time (in seconds) for offset storage in Redis.</td></tr><tr><td>redis_tls_enabled</td><td>bool</td><td>false</td><td>false</td><td>Whether to enable TLS for the Redis connection.</td></tr><tr><td>redis_tls_ca_file</td><td>string</td><td></td><td>false</td><td>The path to the CA file for Redis TLS connection.</td></tr><tr><td>redis_tls_cert_file</td><td>string</td><td></td><td>false</td><td>The path to the client certificate file for Redis TLS connection.</td></tr><tr><td>redis_tls_key_file</td><td>string</td><td></td><td>false</td><td>The path to the client key file for Redis TLS connection.</td></tr></tbody></table>

#### Example Configuration <a href="#example-configuration" id="example-configuration"></a>

<figure><img src="https://1405008107-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FgmiOMzBfoNFwmKJFHMcJ%2Fuploads%2FpyNwBCxwZgpDqUVgPx9K%2FScreenshot%202026-03-13%20at%208.23.55%E2%80%AFPM.png?alt=media&#x26;token=f3a75daf-35ca-44c0-9cab-79852376d253" alt=""><figcaption></figcaption></figure>

#### Component Telemetry <a href="#component-telemetry" id="component-telemetry"></a>

This component emits telemetry that can provide insight into how it is performing. The collector is configured to emit these metrics to `localhost:8888/metrics` by default.

<table><thead><tr><th>Metric Name</th><th width="162.04296875">Type</th><th>Description</th></tr></thead><tbody><tr><td>otelcol_gcsevent.batch_size</td><td>histogram</td><td>The number of logs in a batch.</td></tr><tr><td>otelcol_gcsevent.objects_handled</td><td>counter</td><td>The number of GCS objects processed by the receiver.</td></tr><tr><td>otelcol_gcsevent.failures</td><td>counter</td><td>The number of failures encountered while processing GCS objects.</td></tr><tr><td>otelcol_gcsevent.parse_errors</td><td>counter</td><td>The number of individual log records skipped due to parse errors within a GCS object.</td></tr><tr><td>otelcol_gcsevent.dlq_file_not_found_errors</td><td>counter</td><td>The number of file not found errors that triggered DLQ processing.</td></tr><tr><td>otelcol_gcsevent.dlq_iam_errors</td><td>counter</td><td>The number of IAM permission denied errors that triggered DLQ processing.</td></tr><tr><td>otelcol_gcsevent.dlq_unsupported_file_errors</td><td>counter</td><td>The number of unsupported file type errors that triggered DLQ processing.</td></tr></tbody></table>