Experiments
The Experiments panel in Debugger mode turns any generation run into aportable, reproducible artifact. An experiment is a JSON file containing the exact prompt, model revision, decoding settings, any intervention (patch, ablation), and the measured results — so another machine can replay or re-verify the exact same run.
What is captured
Each experiment (schema version 1) records:
- Kind —
generation,patch, orablation. - Model —
id,revision(the resolved commit),runtime, anddevice. - Input — the prompt and the full option set: max new tokens (1–256), decoding mode (greedy / sampling / sliding_window / speculative), top-k, temperature, top-p, seed, window size, draft gamma, the trace flag, and any source sentence, patched layers, zeroed heads/layers, or needle.
- Results — the measured output, including the optional full trace.
Exported files are named tokenprint-<kind>.experiment.json and carry aSHA-256 checksum over the canonical JSON, computed with crypto.subtle.digest. The checksum detects accidental changes to the file.
Workflow
The panel offers the following actions:
| Action | What it does |
|---|---|
| Save current generation | Captures the current generation (or a loaded recorded demo) as an experiment and downloads it immediately. |
| Import experiment | Loads a .json experiment file after passing the integrity check. Importing never runs inference. |
| Export experiment | Downloads the selected experiment file. |
| Export results | Downloads just the measured results as experiment-results.json. |
| Replay saved results | Loads the recorded trace as a replay. Open Generation to step through it. |
| Re-run and verify | Re-runs the saved experiment and compares measurements against the recorded results. |
| Cancel verification | Aborts an in-progress re-run. |
Verification requires a recorded seed for sampling runs and a valid 40-hex model revision, and compares values to a 1e-4 tolerance. A re-run that matches within tolerance confirms the experiment is reproducible.
Storage and limits
- Experiments are held in memory for the session (up to 10 items). They do not persist across reloads — the downloaded JSON file is the durable copy.
- Imports are capped at 32 MB (the trace import limit).
- Verification has a 5-minute timeout and a
1e-4numerical tolerance.
Guided replay
When a recorded demo trace is loaded with ?tour=1 in the URL, a RECORDED REPLAY guide appears: a 3-step tour (Follow a token, Inspect a layer, Explore the evidence) with a step counter and Next step/Finish tour/Skip tour controls. Replays are a playback of captured values — nothing is re-inferred during playback.
The re-run verification path uses the in-browser GPT-2 engine when available, so a saved experiment can be verified even without the Python backend. See Browser GPT-2.