03 Execute migration — instructor notes

Timing

This is the longest module. Pause after target schema creation, after dual-write starts, and before cutover. Do not let a fast group skip validation while another group is still ingesting.

Teaching points

• Create dictionaries before tables because materialized columns resolve dictionary references at DDL time.
• The Null table plus materialized view is an insert-time transformation boundary, not duplicate storage.
• The collector fan-out is the migration control plane: the same live event reaches both backends.
• AggregatingMergeTree materialized views replace periodic Elasticsearch transforms with incremental states.

Stop-the-room conditions

• Collector logs contain ClickHouse connection failures.
• Elasticsearch and ClickHouse counts diverge outside the documented tolerance.
• Enrichment validation fails.
• Any learner is about to stop Elasticsearch before the final validation scripts pass.

Reset

Run bash common/cleanup.sh from the artifact root, then restart at module 01. For a collector-only failure, remove otelcol-lab, leave the source stack running, and repeat Step 3.

Hard checkpoint

Both collectors use cutover configs, Elasticsearch is stopped, ClickHouse counts continue to grow, and the learner can search correlated telemetry in HyperDX.