Snowflake MigrationClickHouse Workshops

00 Setup

Install the toolchain, create the two cloud trial accounts, clone the repo, and build both dbt virtualenvs — everything the migration needs before you touch data.

Starting point

Nothing installed yet. You need a terminal and about 30 minutes: installing five command-line tools, signing up for two free cloud trials, cloning the repo, and building two Python virtualenvs.

Why

This workshop provisions two clouds — Snowflake as the source, ClickHouse Cloud as the target — and runs two dbt adapters against them, one per cloud. Both adapters have hard version floors, and the one that actually bites is Python: dbt-snowflake (and dbt-clickhouse, used later) requires Python 3.11, 3.12, or 3.13. Python 3.14+ breaks a transitive dependency, mashumaro, that both adapters rely on. Get this wrong now and it does not fail here — it surfaces two modules later as a confusing dbt import error that has nothing obviously to do with your Python version. Pinning the interpreter in this module is what makes that failure not happen.

Step 1 — Install the toolchain

ToolVersionInstall
Terraform>= 1.6Provisions Snowflake and ClickHouse Cloud infrastructure
Docker Desktop>= 24Runs the trip producer and Superset
Python3.11–3.13Migration scripts, dbt, and utility scripts — see the callout below
dbt Core>= 1.8Snowflake and ClickHouse pipelines
SnowSQL CLI>= 1.2Runs SQL against Snowflake from the terminal

Python version matters. dbt-snowflake and dbt-clickhouse both require Python 3.11, 3.12, or 3.13. Python 3.14+ breaks their shared mashumaro dependency. If your system Python is already 3.14+, install 3.13 alongside it (for example brew install python@3.13) rather than replacing your default — Step 4 below uses the python3.13 interpreter explicitly for exactly this reason.

Every package you install for this workshop goes into an isolated virtualenv, never a bare global pip install — Step 4 sets up the two you need.

Step 2 — Create the two cloud accounts

You need one trial account in each cloud, both free with no credit card:

  • Snowflake trial account — this is the source environment. Module 01 provisions a warehouse, database, and Medallion pipeline here.
  • ClickHouse Cloud trial account — this is the target environment. It stays untouched until module 03 provisions it and the migration begins in earnest.

Signing up for both now means neither account is a blocker once you start module 01.

Step 3 — Clone the repository

git clone https://github.com/ClickHouse/ClickHouse_Demos.git
cd ClickHouse_Demos
cd "$(git rev-parse --show-toplevel)/workshop_public/snowflake_migration_lab"

Every command in the rest of this workshop assumes you are standing in workshop_public/snowflake_migration_lab inside this clone.

Step 4 — Build the two dbt virtualenvs

dbt-snowflake and dbt-clickhouse pin conflicting dependency ranges, so they cannot share one virtualenv — you build one per adapter, one per cloud side of the migration.

dbt-snowflake, used against the source in module 01:

cd "$(git rev-parse --show-toplevel)/workshop_public/snowflake_migration_lab/01-setup-snowflake"

# Use python3.13 explicitly if your system default is 3.14+
python3.13 -m venv .venv          # or: python3 -m venv .venv
source .venv/bin/activate
pip install "dbt-snowflake>=1.7,<2.0"
deactivate

dbt-clickhouse, used against the target starting in module 03:

cd "$(git rev-parse --show-toplevel)/workshop_public/snowflake_migration_lab/03-migrate-to-clickhouse"
python3.13 -m venv .venv
source .venv/bin/activate
pip install "dbt-clickhouse>=1.8,<2.0" snowflake-connector-python clickhouse-connect
deactivate

Each .venv stays inside its own module directory — you activate the one you need and deactivate when you switch, rather than trying to make one environment satisfy both adapters.

How to verify you are done

cd "$(git rev-parse --show-toplevel)/workshop_public/snowflake_migration_lab"

terraform version                  # expect >= 1.6
docker --version                   # expect >= 24
python3 --version                  # expect 3.11, 3.12, or 3.13
snowsql --version                  # expect >= 1.2

source 01-setup-snowflake/.venv/bin/activate
dbt --version                      # expect dbt-snowflake >= 1.7
deactivate

source 03-migrate-to-clickhouse/.venv/bin/activate
dbt --version                      # expect dbt-clickhouse >= 1.8
deactivate

All five commands should print a version at or above the floor shown, and each dbt --version should list the adapter you just installed into that venv.

End state

The toolchain is installed, both trial accounts are live, the repo is cloned, and both dbt virtualenvs are built. Nothing is provisioned in either cloud yet — that starts in module 01. Continue to 01 Source environment to build the Snowflake side of the migration.

On this page