Learner track
The hands-on lessons you work through during the session.
This is the Learner track: the hands-on lessons you work through during the workshop. Each module builds on the last, taking the AgentArena NL→SQL chatbot through one flow — select the base model → measure its quality → continuously improve → release to production — all instrumented with LangFuse from the very first module, served over OpenRouter, and backed by ClickHouse.
The arc starts with the decision that matters most when building a serious agent: which model to use. Module 01 answers it with evidence — the contest, run as LangFuse experiments and ranked by cost per correct answer — and every module after it builds on that choice.
Work through the modules in order:
- 00 Setup — connect OpenRouter, ClickHouse Cloud, and LangFuse Cloud, and seed the database.
- 01 Select the base model — the foundational decision: run the model × prompt grid as LangFuse experiments and crown a winner by cost per correct answer.
- 02 Measure quality — go beyond "it won": read per-tier
accuracy, the
llm_judgesignal, and individual traces to see how the winner really performs. - 03 Improve continuously — turn reviewed feedback into new golden questions and feed them back into the Arena.
- 04 Release to production — run the winning configuration live behind a real API, and watch it with the same LangFuse project.
See the top-level overview for the full module table, including the matching Instructor-track notes for each module.