> For the complete documentation index, see [llms.txt](https://docs.bindplane.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bindplane.com/integrations/destinations/snowflake.md). # Snowflake The Snowflake destination exports logs, metrics, and traces from a Bindplane pipeline to Snowflake, a cloud data warehouse. The exporter authenticates with a Snowflake user and writes telemetry into tables under a database, schema, and warehouse you specify. Databases, schemas, and tables are created automatically if they do not already exist, provided the configured user has the required privileges. ### Supported Telemetry | Metrics | Logs | Traces | | ------- | ---- | ------ | | ✓ | ✓ | ✓ | ### Prerequisites You need the following on the Snowflake side before configuring this destination: * A Snowflake account, identified by its [account identifier](https://docs.snowflake.com/en/user-guide/admin-account-identifier). * A Snowflake [warehouse](https://docs.snowflake.com/en/user-guide/warehouses-overview) to run the writes against. * A user and role for the collector to authenticate as. The role needs `USAGE` on the warehouse, `CREATE DATABASE` on the account (so the destination can create the configured database, schemas, and tables), or the equivalent privileges on a pre-created database. See [Granting Snowflake Privileges](https://github.com/observIQ/bindplane-otel-collector/tree/main/exporter/snowflakeexporter#granting-snowflake-privileges) in the exporter documentation. A minimal grant against one of Snowflake's default roles (`SYSADMIN`) looks like the following. Run these with a role that can grant privileges and create users, such as `ACCOUNTADMIN`, replacing the warehouse and user values with your own: ```sql -- Allow the role to use the warehouse and create databases GRANT USAGE ON WAREHOUSE "TEST" TO ROLE "SYSADMIN"; GRANT CREATE DATABASE ON ACCOUNT TO ROLE "SYSADMIN"; -- Create the collector user and assign the role CREATE USER BP_AGENT PASSWORD="password" LOGIN_NAME="BP_AGENT" DEFAULT_ROLE="SYSADMIN"; GRANT ROLE "SYSADMIN" TO USER BP_AGENT; ``` If the collector user's default role does not hold these privileges, set the **Role** parameter so the exporter assumes one that does. For alternative grant approaches, see the exporter's [alternatives](https://github.com/observIQ/bindplane-otel-collector/tree/main/exporter/snowflakeexporter#alternatives) section. ### Configuration
Bindplane docs - Snowflake - image 1
**Account Information** | Parameter | Type | Required | Default | Description | | --------------------- | ------------------ | -------- | --------------------- | ---------------------------------------------------------------------------------------------------- | | Choose Telemetry Type | Telemetry Selector | Yes | Logs, Metrics, Traces | Which signals to export to Snowflake. Controls which of the schema and table parameters below apply. | | Account Identifier | String | Yes | *(empty)* | The account identifier for the Snowflake account that data is sent to. | | Warehouse | String | Yes | *(empty)* | The Snowflake warehouse that telemetry data is stored in. | | Username | String | Yes | *(empty)* | Username the exporter uses to authenticate with Snowflake. | | Password | String (sensitive) | Yes | *(empty)* | Password the exporter uses to authenticate with Snowflake. | | Database | String | Yes | `otlp` | The Snowflake database that telemetry schemas are stored in. Created if it does not exist. | **Logs** Applies when **Logs** is selected. | Parameter | Type | Required | Default | Description | | ---------- | ------ | -------- | ------- | -------------------------------------------------------------------- | | Log Schema | String | No | `logs` | The schema the log table is stored in. Created if it does not exist. | | Log Table | String | No | `data` | The table logs are stored in. Created if it does not exist. | **Metrics** Applies when **Metrics** is selected. | Parameter | Type | Required | Default | Description | | ------------- | ------ | -------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Metric Schema | String | No | `metrics` | The schema metric tables are stored in. Created if it does not exist. | | Metric Table | String | No | `data` | The prefix used for metric tables. Tables are created if they do not exist. See the exporter [metrics](https://github.com/observIQ/bindplane-otel-collector/tree/main/exporter/snowflakeexporter#metrics) documentation. | **Traces** Applies when **Traces** is selected. | Parameter | Type | Required | Default | Description | | ------------ | ------ | -------- | -------- | ---------------------------------------------------------------------- | | Trace Schema | String | No | `traces` | The schema the trace table is stored in. Created if it does not exist. | | Trace Table | String | No | `data` | The table traces are stored in. Created if it does not exist. | **Advanced** | Parameter | Type | Required | Default | Description | | ------------- | ------- | -------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Role | String | No | *(empty)* | The role the exporter should use. Required if the default role of the provided credentials does not have the correct privileges. | | Parameters | Map | No | *(empty)* | Additional optional [parameters](https://docs.snowflake.com/en/sql-reference/parameters) the exporter uses when connecting to Snowflake. Generally not required. | | Drop Raw Copy | Boolean | No | `true` | When enabled, the raw copy of the log stored in `log.record.original` is dropped before export. Logs only. | **Retry on Failure** | Parameter | Type | Required | Default | Description | | ----------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | Enable Retry on Failure | Boolean | No | `true` | Attempt to resend telemetry data that failed to transmit to the destination. | | Initial interval | Integer | No | `5` | Time (in seconds) to wait after the first failure before retrying. Applies when retry is enabled. | | Max interval | Integer | No | `30` | Upper bound (in seconds) on backoff. Applies when retry is enabled. | | Max elapsed time | Integer | No | `300` | Maximum time (in seconds) spent trying to send a batch before giving up, to avoid a never-ending retry loop. Applies when retry is enabled. | **Sending Queue** | Parameter | Type | Required | Default | Description | | ------------------------- | --------- | -------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Enable Sending Queue | Boolean | No | `true` | Buffer telemetry data temporarily before sending to help ensure data is not lost during a temporary network outage. | | Number of Consumers | Integer | No | `10` | Number of consumers that dequeue batches. Applies when the sending queue is enabled. | | Queue Size | Integer | No | `5000` | Maximum number of batches kept in memory before dropping. Applies when the sending queue is enabled. | | Enable Persistent Queuing | Boolean | No | `true` | Buffer telemetry data to disk before sending to help ensure data is not lost during network outages or collector restarts. Applies when the sending queue is enabled. | | Persistent Queue Storage | Extension | Yes | `file_storage_persistent_queue` | The storage extension used for the persistent queue. Applies when both the sending queue and persistent queuing are enabled. See [persistent queue](https://docs.bindplane.com/configuration/bindplane-otel-collector/persistent-queue). | ### Examples Send all three telemetry types to a Snowflake account using the default database, schema, and table names, with the sending queue, persistent queue, and retry on failure enabled. The database (`otlp`), schemas (`logs`, `metrics`, `traces`), and tables (`data`) are created automatically if they do not exist. ```yaml apiVersion: bindplane.observiq.com/v1 kind: Destination metadata: name: snowflake spec: type: snowflake parameters: - name: telemetry_types value: - Logs - Metrics - Traces - name: account_id value: my_account_id - name: warehouse value: my_warehouse - name: username value: BP_AGENT - name: password value: my_password - name: database value: otlp - name: sending_queue_enabled value: true - name: persistent_queue_enabled value: true - name: retry_on_failure_enabled value: true ``` ### Configuration Tips * Leave **Database**, the schema fields, and the table fields at their defaults to get telemetry flowing quickly. The exporter creates each resource on first write if the configured user has `CREATE DATABASE` (and the implied schema and table creation) privileges. * Use **Role** when the collector user's default role lacks the required grants. This avoids granting broad privileges to the user's default role and lets the exporter assume a purpose-built role instead. * Keep **Drop Raw Copy** enabled to avoid storing a duplicate, full copy of each log in `log.record.original`, which reduces table size and storage cost in Snowflake. Disable it only if you need the unmodified original record. * Keep the persistent queue enabled so buffered telemetry survives collector restarts. The default storage extension writes to `${OIQ_OTEL_COLLECTOR_HOME}/storage`. ### Troubleshooting #### Authentication or permission errors Symptoms: the exporter logs authentication failures, or errors creating the database, schema, or table. Solutions: 1. Verify the **Account Identifier**, **Username**, and **Password** are correct. 2. Confirm the user's role holds `USAGE` on the warehouse and `CREATE DATABASE` on the account, or the equivalent privileges on a pre-created database. See [Granting Snowflake Privileges](https://github.com/observIQ/bindplane-otel-collector/tree/main/exporter/snowflakeexporter#granting-snowflake-privileges). 3. If the user's default role lacks these privileges, set the **Role** parameter to a role that has them. #### Warehouse not found or suspended Symptoms: writes fail with a warehouse error, or data does not appear in Snowflake. Solutions: 1. Confirm the **Warehouse** name matches an existing warehouse exactly, including case where the warehouse was created with a quoted identifier. 2. Ensure the configured role has `USAGE` on that warehouse and that the warehouse is able to resume to run the writes. #### Telemetry is delayed or dropped under load Symptoms: data lags behind, or the collector reports dropped batches. Solutions: 1. Keep **Enable Retry on Failure** on so transient Snowflake or network errors are retried. 2. Keep **Enable Sending Queue** and **Enable Persistent Queuing** on so batches are buffered rather than dropped during outages or restarts. 3. If the in-memory queue fills, increase **Queue Size** or **Number of Consumers** to raise throughput. ### Standalone Destination ```yaml apiVersion: bindplane.observiq.com/v1 kind: Destination metadata: name: snowflake spec: type: snowflake parameters: - name: telemetry_types value: - Logs - Metrics - Traces - name: account_id value: my_account_id - name: warehouse value: my_warehouse - name: username value: BP_AGENT - name: password value: my_password - name: database value: otlp ``` ### Related Resources * [Snowflake exporter (bindplane-otel-collector)](https://github.com/observIQ/bindplane-otel-collector/tree/main/exporter/snowflakeexporter) * [Snowflake account identifiers](https://docs.snowflake.com/en/user-guide/admin-account-identifier) * [Snowflake warehouses overview](https://docs.snowflake.com/en/user-guide/warehouses-overview) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.bindplane.com/integrations/destinations/snowflake.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.