> ## Documentation Index
> Fetch the complete documentation index at: https://www.propeldata.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Snowflake to ClickHouse
> Ingest data from Snowflake.
Ingest data from Snowflake tables, views, or Dynamic Tables to Propel.
Step-by-step instructions to connect your Snowflake data warehouse to Propel.
## Architecture
Snowflake Data Pools connect to your Snowflake instance to sync data from a specified table, view, or Dynamic Table into Propel.
Syncs can be configured to occur at intervals ranging from 1 minute to 24 hours.
## Features
| Feature name | Supported | Notes |
| ------------------------------------ | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Syncs inserts, updates, and deletes. | ✅ | See [How Propel syncs inserts, updates, and deletes](#how-propel-syncs-inserts-updates-and-deletes). |
| Re-sync | ✅ | See [Re-sync](#re-syncing-data) section. |
| Configurable sync interval | ✅ | See [How Propel syncs](#how-propel-syncs-inserts-updates-and-deletes) section. It can be configured to occur at intervals ranging from every minute to every 24 hours. |
| Sync Pausing / Resuming | ✅ | |
| Real-time updates | ✅ | See the [Real-time updates](/docs/updating-data#real-time-updates) section. |
| Real-time deletes | ✅ | See the [Real-time deletes](/docs/deleting-data#real-time-deletes) section. |
| Batch Delete API | ✅ | See [Batch Delete API](/docs/deleting-data#batch-deletes). |
| Batch Update API | ✅ | See [Batch Update API](/docs/updating-data#batch-updates). |
| API configurable | ✅ | See [API](/docs/management-api) docs. |
| Terraform configurable | ✅ | See [Terraform](/docs/terraform) docs. |
## How Propel syncs inserts, updates, and deletes
Propel uses [Snowflake streams](https://docs.snowflake.com/en/user-guide/streams-intro) for change data capture to sync records. The sync behavior depends on your chosen table engine:
* **MergeTree (append-only):** Syncs inserts only.
* **ReplacingMergeTree (mutable):** Syncs inserts, updates, and deletes based on sorting key.
See our [table engine guide](/docs/guides/table-engine-and-sorting-key) for details.
Sync intervals range from 1 minute to 24 hours. Choose based on your data freshness needs and upstream pipeline frequency.
Snowflake warehouse activates on each sync. Consider costs when setting intervals. We recommend a 60-second minimum auto-suspend period.
## Supported data structures
Propel can sync these Snowflake data structures:
| Structure | Requirement |
| -------------- | ---------------------------------------------------- |
| Table | Change tracking enabled |
| View | Change tracking enabled on underlying tables |
| Dynamic Tables | No additional setup (change tracking always enabled) |
## Schema changes
Propel enables the addition of new columns to Snowflake Data Pools through the `AddColumnToDataPool` job.
For breaking changes like column deletions or type modifications, recreate the Data Pool.
See our [Changing Schemas](/docs/changing-schemas) section for more details.
## Re-syncing data
Snowflake Data Pools with ReplacingMergeTree engine support safe re-syncing of all data. This operation:
* Continues serving data to your application
* Avoids creating duplicates
* Doesn't persist deleted Snowflake data
Re-sync when:
* Snowflake stream is deleted, failed, or stale
* Table is re-created
* Stream is manually deleted
* `dbt full-refresh` operation re-creates the table
During re-sync, Propel recreates the Snowflake stream on the table or view.
Use the `reSyncSnowflakeDataPool` mutation for API-based re-syncing.
## Data Types
The table below shows default Snowflake to Propel data type mappings. When creating a Snowflake Data Pool, you can customize these mappings.
| Snowflake Type | Propel Type | Notes |
| ------------------------------------------------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DATETIME`, `TIMESTAMP_NTZ`, `TIMESTAMP_LTZ`, `TIMESTAMP_TZ`, `TIMESTAMP(p)` | `TIMESTAMP` | Timestamps without a timezone will be synced as UTC. |
| `DATE` | `DATE` | |
| `BOOLEAN` | `BOOLEAN` | |
| `NUMBER(p≤9, s=0)` | `INT32` | |
| `NUMBER(p≤18, s=0)` | `INT64` | |
| `NUMBER(p≤9, s>0)` | `FLOAT` | |
| `NUMBER(p≤18, s>0)`, `NUMBER(p>18, s)` | `DOUBLE` | |
| `NUMBER(p, s)`, `DECIMAL(p, s)`, `NUMERIC(p, s)` | `Depends on precision and scale` | |
| `NUMBER(38, s>0)`, `INT`, `INTEGER`, `BIGINT`, `SMALLINT`, `TINYINT`, `BYTEINT` | `DOUBLE` | Snowflake maps INT types to `NUMBER(38, 0)`, which Propel represents as a `DOUBLE` as it can have numbers larger than `INT64`. See [Snowflake numeric types](https://docs.snowflake.com/en/sql-reference/data-types-numeric). |
| `FLOAT`, `FLOAT4`, `FLOAT8`, `DOUBLE`, `DOUBLE PRECISION`, `REAL` | `DOUBLE` | |
| `VARCHAR`, `CHAR`, `CHARACTER`, `STRING`, `TEXT`, `BINARY`, `VARBINARY` | `STRING` | |
| `TIME` | `STRING` | |
| `VARIANT`, `OBJECT`, `ARRAY` | `JSON` | |
| `GEOGRAPHY`, `GEOMETRY` | Not supported | |
## Frequently Asked Questions
Data is synchronized based on your configured sync interval. You can sync as often as every minute or up to 24 hours. Once a sync is initiated, the data will be available in Propel and served via the API in 1-3 minutes.
No. Propel only queries Snowflake to sync newly inserted, updated, or deleted records.
Yes. When you create the Data Pool, you can select which columns from the underlying table you want to sync. This is useful if there is PII or any other data that you don't need in Propel.
Even if the Snowflake is down, Propel will continue to serve data via [SQL](/docs/sql-reference) and the [Data API](/docs/query-apis). New data will not sync until the data store comes back online.
The syncing interval starts when your Data Pool goes `LIVE` or when syncing is resumed.