Your First Trace

Getting Started

A trace is a full record of a single forward pass — every operation in every layer, in order. TokenPrint records traces in real time as the model runs and replays them as 3D animations.

What a trace contains

Each trace frame records:

The current operation name (e.g., attn.q_proj)
The layer index (0–23 for the reference model)
The KV cache phase (prefill or decode)
Timing metadata (host-side, not GPU-synchronized)
A reference to the tensor being produced

The op catalog for the reference model contains 243 operations in a single forward pass.

Reading the animation

In Generation mode, the 3D model animates as the trace plays. The component currently executing is highlighted. The right panel shows the operation name, layer, input/output shapes, and parameter count.

The operation order within each layer follows the actual computation graph:

Input RMSNorm (input_layernorm)
Q projection (attn.q_proj)
K projection (attn.k_proj)
V projection (attn.v_proj)
Attention (attention)
Output projection (attn.o_proj)
Post-attention RMSNorm (post_attention_layernorm)
MLP gate projection (mlp.gate_proj)
MLP up projection (mlp.up_proj)
MLP down projection (mlp.down_proj)

Note

This order is not configurable — it reflects the actual PyTorch execution order captured by register_forward_hook on each module.

KV cache phases

The trace distinguishes two phases, shown as a badge in the transport bar:

Phase	What happens	Positions computed
`prefill`	The full prompt is processed in one pass. The KV cache is built.	All prompt tokens (e.g., 39 for a 39-token prompt)
`decode`	One new token is generated per step. Only the new token position is computed; all prior KV pairs are read from cache.	1 (the new token)

Replay and branching

Completed traces are saved and can be replayed from the Debugger mode. The trace branching panel lets you re-run a trace with a different prompt or model configuration and compare the two side by side.