BigQuery MigrationClickHouse Workshops

02 Query it in place

Read the GA4 export directly out of object storage with no ingestion at all, and feel why that is not how you want to run a dashboard.

Nothing in this module depends on anything you did in module 01. If you are catching up, or your account or service is still being set up, this module needs only a working SQL console -- run the query below the moment you have one.

The bucket

The same GA4 events dataset, exported to Parquet and shifted five years forward in time (more on the shift below), lives in a public Google Cloud Storage bucket that takes anonymous reads -- no credentials of any kind:

https://storage.googleapis.com/ch-workshop-bq-migration/ga4-events/*.parquet

Use gcs(), not url(), to read it. gcs() lists the bucket itself, so it can expand the * glob across every object; url() fetches a single HTTP resource and cannot glob at all, so the same wildcard silently does nothing useful with it.

About that five-year shift

The export's dates are shifted five years forward from the public dataset you queried in module 01 -- 2025-11-01 through 2026-01-31, instead of 2020-11-01 through 2021-01-31 -- so that dates like 11.11 and 12.12 land inside the window. user_pseudo_id is not shifted, so the probe user from module 01 exists identically here. Both copies are correct; if you compare a date between the two you are comparing a shifted copy against an unshifted one, not spotting a bug.

Read the schema BigQuery exported

DESCRIBE TABLE gcs('https://storage.googleapis.com/ch-workshop-bq-migration/ga4-events/*.parquet', NOSIGN, 'Parquet');

ClickHouse infers a column list straight from the Parquet metadata: every column comes back Nullable, event_date is a String, event_timestamp is a raw Int64 of microseconds, and the nested fields (event_params, items, device, geo, ecommerce, traffic_source) come back as arrays of tuples and tuples of tuples. Nothing here has been reshaped for the lab -- this is what BigQuery's own export looks like. Module 03 turns this same inference into a CREATE TABLE.

Count it

SELECT count()
FROM gcs('https://storage.googleapis.com/ch-workshop-bq-migration/ga4-events/*.parquet', NOSIGN, 'Parquet');

Expect 4,295,584 -- the full export, read with no table, no database, and no ingestion step of any kind.

A first funnel query

SELECT event_name, count() AS n
FROM gcs('https://storage.googleapis.com/ch-workshop-bq-migration/ga4-events/*.parquet', NOSIGN, 'Parquet')
WHERE event_name IN ('view_item', 'add_to_cart', 'begin_checkout', 'purchase')
GROUP BY event_name
ORDER BY n DESC;

Expect this back, in the same order the ORDER BY n DESC puts it in -- the same conversion funnel this workshop keeps coming back to:

event_namen
view_item386,068
add_to_cart58,543
begin_checkout38,757
purchase5,692

Timing will vary with your service's size and the network path to the bucket, but for reference, one run against a modest service answered quickly even reading Parquet directly over the network:

0.178selapsed, one run
1 replica, 12 GiBservice size

Why this is not how you want to run a dashboard

Every query above re-reads and re-parses the same Parquet objects over the network, from scratch, every single time -- there is no local copy, no cache, and no index behind any of it. Run the count query twice in a row and both runs pay that cost again. This is a genuinely useful way to explore data you have not committed to ingesting yet, and it is real, working ClickHouse SQL against object storage. It is not, however, what you want standing behind a dashboard that real users hit repeatedly. The next module gives ClickHouse a copy of the same rows to work with.

Done when

DESCRIBE TABLE returned a column list with no table behind it, count() returned 4,295,584, and the funnel query's four counts matched the figures above. Continue to 03 The lazy migration.

本页内容

Track your progress?

Optional. We email a link to confirm your address; progress records once you open it.

Please use your work email address, not a personal one.

Progress tracking also requires accepting the current Terms of Service in Privacy settings.

ZH