Logits
After the final RMSNorm, the residual stream is projected through the unembedding matrix to produce a vector of 151,936 logits — one real number per vocabulary token. These logits are the raw scores before any normalization. The model's prediction is the token with the highest logit.
Unembedding
logits = h_final @ embed_matrix_T # [T, vocab_size] = [T, 151936]
In Qwen2.5, the unembedding matrix is the transpose of the embedding matrix (weight tying). The same 136,134,656 parameters serve both input lookup and output projection.
Logit lens
The logit lens applies the final unembedding matrix to the residual stream at every intermediate layer, producing a probability distribution at each layer. This reveals how the model's prediction evolves through the depth of the network.
TokenPrint computes the full logit lens matrix as part of POST /analyze — shape [n_layers, n_positions, top_k] where each entry is a (token_text, token_id, probability) triple. This data is what drives the Walkthrough Softmax & Output chapter.
Top-k skyline
In Generation mode, the right panel shows a top-k skyline chart — a bar chart of the top-5 predicted tokens at the current step with their real softmax probabilities. This updates live as each token is generated.
The prediction game lets you select your guess from the top candidates before the model reveals its answer — it is scored against the real argmax, not a simulation.