Browser GPT-2
TokenPrint can run a real GPT-2 forward pass entirely in the browser — no Python backend, no WebSocket. The generation runs in a module Web Worker using Transformers.js, with WebGPU or CPU (WASM) as the compute device.
The browser engine is fully client-side and works against the static production export. You can deploy the frontend alone and still run generation.
Selecting the engine
In Generation mode, the Inference engine selector offers three options:
Python backend— the default, streams overWS /ws/generate.Browser GPT-2 · WebGPU— runs on the GPU via WebGPU.Browser GPT-2 · CPU (WASM)— runs on the CPU via WASM.
When a browser engine is selected, the helper text notes the constraints: greedy decoding, at most 32 new tokens, and a ~500 MB cached download. The transport bar shows a BROWSER · WEBGPU · GPT-2 badge.
Requirements
- WebGPU requires a browser/device with a working GPU adapter and a secure context (HTTPS or localhost).
- CPU (WASM) is an explicit choice — the app never silently switches data sources. If the selected device cannot execute, generation fails rather than falling back.
- The first run downloads ~500 MB of model files. Progress is visible; Stop cancels the worker, including an in-progress download.
Limits
| Constraint | Value |
|---|---|
| Model | Xenova/gpt2 |
| Revision | bf2c7f02e0b826c60d03af341171bde20893da66 (pinned, immutable) |
| Runtime | Transformers.js 3.8.1, ONNX model.onnx, fp32 |
| Decoding | Greedy only |
| New tokens | At most 32 |
| Total window | Prompt + output ≤ 128 tokens |
| Download | ~500 MB, cached by the browser |
GPT-2 is a base completion model, not an instruction/chat model. Treat it as a small, fast embodiment of the architecture for exploration, not as a chat assistant.
Capability boundary
| Capability | Browser GPT-2 | Python backend |
|---|---|---|
| Real token IDs, final logits, probabilities | Yes | Yes |
| Greedy generation | Yes | Yes |
| Sampling / advanced decoding | No | Backend-dependent |
| Attention / hidden states / logit lens | Not exposed by this ONNX export | Model-dependent (capability checked) |
| KV-cache instrumentation | No; sequence recomputed | When supported |
| New ablations / activation patching | No | Model-dependent |
| GGUF inference | Not part of this browser release | Optional llama.cpp backend |
No synthetic activation statistics or operation timings are generated by the browser path. Architecture dimensions come from the pinned model config; they describe the model, not a measured execution trace. The initial release uses full precision because the quantized WebGPU output did not pass the reference check.
Verification
The browser engine is validated against an independent ONNX CPU session via node frontend/scripts/browser-reference.mjs, which generates the checked-in three-prompt reference. A Playwright test downloads and runs the real model for both devices, checks the pinned revision, selected and top-five token IDs, and requires probability error below 1e-4. The test fails if the selected device cannot execute — browser support depends on actual adapter availability.
cd frontend
TOKENPRINT_BROWSER_INFERENCE=1 npx playwright test tests/visual/browser-inference.spec.ts --project=chromiumHow it runs
The engine is registered via registerLocalEngine and executes in a dedicated module Web Worker. Generation is requested with source: "local", a device (webgpu or wasm), decodingMode: "greedy", and the new-token cap clamped to 32. Frames flow through the same frame-producer seam as the live WebSocket, so replay, browser, and backend sources are interchangeable without store changes.