00 Setup

Outcome

A cloned repo with a Python virtualenv installed, a .env populated with credentials for OpenRouter, ClickHouse Cloud, and LangFuse Cloud, an arena database in ClickHouse Cloud seeded with synthetic e-commerce data, and the local dashboard running at http://localhost:5174 — Leaderboard tab empty for now, that's expected until Module 01.

Why

AgentArena is one NL→SQL agent core (agents/) reused by two callers — the benchmark harness (eval/harness.py) and the live serving API (serving/api.py) — so the demo and the benchmark share exactly the same code path: same prompt templates, same model client, same read-only SQL sandbox. That's what makes the benchmark's numbers trustworthy predictors of production behavior, rather than a separate "eval harness" that quietly diverges from what actually ships.

Notice that LangFuse is one of the three accounts you set up in this very first module — before you've picked a model, before you've run a single question. That's deliberate: LangFuse isn't something you bolt on once the chatbot works, it's the tool that runs the contest in Module 01, measures the winner's quality in Module 02, drives the improvement loop in Module 03, and watches production in Module 04 — one project, one set of traces and datasets, start to finish. Every module after this one builds on that one code path and that one LangFuse project, so getting the three accounts and the seeded database right here is what makes the rest of the workshop just work.

Concepts — under the hood

Three pillars, three jobs, wired together from this first module:

• OpenRouter — one OpenAI-compatible API in front of every model family in this workshop (Anthropic, OpenAI, Google, DeepSeek, Qwen, Z.ai). Instead of juggling six provider SDKs and six sets of credentials, the agent core (agents/) speaks one client to reach all six models in config.yaml. That's what makes a fair, apples-to-apples contest in Module 01 possible: every model is one model= string away, behind the same endpoint, the same request shape.
• ClickHouse — the warehouse, doing two jobs at once. It holds the business data (the synthetic e-commerce tables you're about to seed) behind the v_* views, and it holds the results of every benchmark run (eval_runs). The agent is only ever allowed to SELECT from the v_* views — never the raw tables, never write — both because that's a stable, documented read-only contract for the agent to reason about, and because agents/sqlguard.py enforces it as a SELECT-only sandbox: it parses the generated SQL and rejects anything that isn't a single SELECT/WITH…SELECT statement, and blocks a denylist of write/DDL keywords (INSERT, UPDATE, DELETE, DROP, ALTER, SYSTEM, …) even inside an otherwise-valid statement.
• LangFuse — the eval and observability store. It's connected now, before you've picked a model or asked a single question, because it isn't a bolt-on for later — it's the tool that grades the contest in Module 01, lets you drill into quality in Module 02, and watches production in Module 04. One project, one continuous trail of traces and datasets, from the first model choice onward.

Goal

Three sets of credentials in .env, a seeded arena database with the v_* views the agent will query, and the local dashboard reachable in a browser.

Step 1 — Create three accounts

You need API credentials from three services before you touch the terminal:

Keep all three handy — you'll paste them into .env in a moment.

Step 2 — Clone the repo

AgentArena lives inside the ClickHouse_Demos monorepo, under workshops/agent_arena. Clone the whole repo, then move into that subdirectory — every command from here on assumes you're standing in it:

git clone https://github.com/ClickHouse/ClickHouse_Demos.git
cd ClickHouse_Demos/workshops/agent_arena

Step 3 — Create a virtualenv and install dependencies

python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt

Step 4 — Configure .env

Copy the example file:

cp .env.example .env

.env

Fill in the values from Step 1 — every value below is empty or a placeholder in .env.example:

# ClickHouse Cloud (data + eval_runs)
export CLICKHOUSE_CLOUD_HOST=xxx.clickhouse.cloud
export CLICKHOUSE_CLOUD_USER=default
export CLICKHOUSE_CLOUD_PASSWORD=
export CLICKHOUSE_CLOUD_DATABASE=arena
export ARENA_RO_PASSWORD=
# OpenRouter (LLM provider)
export OPENROUTER_API_KEY=sk-or-...
export OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# LangFuse Cloud (eval store + tracing)
export LANGFUSE_BASE_URL=https://us.cloud.langfuse.com
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...

What each block is for:

• CLICKHOUSE_CLOUD_* — your ClickHouse Cloud service's admin credentials. Setup uses the admin user once, to create the arena database and a dedicated read-only user (arena_ro) that the agent queries through for the rest of the workshop.
• ARENA_RO_PASSWORD — pick any password; it becomes arena_ro's password when the read-only user is created.
• OPENROUTER_* — your OpenRouter key and its base URL. Every model in config.yaml is addressed through this one endpoint.
• LANGFUSE_* — your LangFuse Cloud project's host and keys. This is where every trace, score, and dataset in the workshop lives, from the first Arena run in Module 01 through production in Module 04.

Step 5 — Seed ClickHouse with synthetic e-commerce data

This is the ClickHouse-only path — it creates the arena database, the arena_ro read-only user, generates synthetic e-commerce data (customers, products, orders, order items, events) directly into ClickHouse, and builds the v_* views the agent queries against:

source .env && scripts/arena.sh up

Every agent, prompt, and piece of golden SQL in this workshop queries the v_customers, v_products, v_orders, v_order_items, and v_events views — never the raw tables.

scripts/arena.sh up also starts the local dashboard API and web UI. When it finishes, open http://localhost:5174 — the Leaderboard tab will be empty until you run the contest in Module 01.

What a healthy run looks like. scripts/arena.sh up prints, in order:

1. ClickHouse: database + read-only user + results tables — the arena database, the arena_ro read-only user, and the empty eval_runs table get created.
2. Seeding ClickHouse directly + views + schema context — one clickhouse: inserted <N> into <table> line per table (customers, products, orders, order_items, events), then done, then the v_* views and the schema context the agent reads get built.
3. Starting dashboard API (:8000) + web UI (:5174) — two [ready] lines. If either says [NOT up], the port is probably already taken; check the log path it prints (.run/dashboard-api.log or .run/web.log).

You can re-check any of this later with scripts/arena.sh status, which prints whether the servers are up and the row count of each v_* view.

How to verify you are done

• .env has real values (not placeholders) for CLICKHOUSE_CLOUD_*, OPENROUTER_*, and LANGFUSE_*.
• scripts/arena.sh up finished without error.
• http://localhost:5174 loads in a browser, showing a Leaderboard tab (empty is correct for now).

Exercise — break and diagnose a connection

Learn to recognize a broken .env value by its failure signature, on purpose, while the stakes are zero:

1. Open .env and change one character in ARENA_RO_PASSWORD (or temporarily comment it out).
2. Re-run source .env && scripts/arena.sh up. It should still get through the ClickHouse admin steps (those use the admin credentials), but watch for where the read-only path — anything that connects as arena_ro — starts complaining.
3. Read the error message closely: is it an authentication error, a "user does not exist" error, or silence followed by a timeout? Note which one you got.
4. Restore the correct ARENA_RO_PASSWORD and re-run scripts/arena.sh up. Confirm it completes cleanly again.

This is the same diagnostic instinct you'll want later when a teammate's setup "isn't working" — matching the error text to which of the three services is misconfigured, instead of re-checking everything.

Wrap-up

You have a seeded ClickHouse database, credentials for all three services — including LangFuse, connected before any model has been chosen — and the local dashboard running. Everything after this module reuses this same environment and this same LangFuse project; no further setup steps.

End state

Environment ready. Continue to 01 Select the base model to run the contest against this data.