GETTING STARTED

Research & Theoretical Background

TokenPrint is grounded in research on transformer mechanistic interpretability, real-time activation tracing, and high-dimensional tensor trajectory visualization.

Conceptual diagram of residual stream propagation and activation flow across transformer layers.

Mechanistic Interpretability

Mechanistic interpretability aims to reverse-engineer trained neural networks into human-understandable algorithms. TokenPrint enables fine-grained inspection of model internals during execution by capturing exact activation values, attention scores, and projection weights without approximations.

Zero-Mock Guarantee

Every tensor value, norm, matrix heat map, and activation vector shown in TokenPrint is derived directly from live forward pass telemetry generated by standard model execution backends.

Core Research Pillars

1. Residual Stream Telemetry

The residual stream serves as the central communication bus of the transformer architecture. TokenPrint visualizes how hidden state representations evolve layer-by-layer as attention heads and MLP blocks write information into the stream.

2. Grouped Query Attention & Key-Value Cache

Modern open LLMs (such as Qwen2.5, LLaMA-3, and Mistral) utilize Grouped Query Attention (GQA) to minimize KV cache footprint during decoding. TokenPrint provides explicit visual breakdowns of Query head to Key/Value head assignments, head-wise entropy, and memory allocation.

3. High-Dimensional Spatial Projections

To render hidden state vectors in 3D, TokenPrint employs real-time dimensionality reduction techniques (including PCA and t-SNE projections) allowing researchers to observe vector trajectories across network depth.

Key References & Literature

TokenPrint builds upon foundational research in transformer architectures and interpretability:

Attention Is All You Need (Vaswani et al., 2017) — arXiv:1706.03762
GQA: Training Generalized Multi-Query Transformer Models (Ainslie et al., 2023) — arXiv:2305.13245
RoFormer: Enhanced Transformer with Rotary Position Embedding (Su et al., 2021) — arXiv:2104.09864
SwiGLU: GLU Variants Improve Transformer (Shazeer, 2020) — arXiv:2002.05202
Transformer Circuits Framework (Elhage et al., Anthropic Research) — Mathematical framework for analyzing transformer activations.

Citing TokenPrint

If you use TokenPrint in your research or technical work, please cite the repository:

bibtex

@misc{tokenprint2026,
  author = {Selvaraj, Sudharsan},
  title = {TokenPrint: Interactive 3D Visual Debugger for LLM & Transformer Internals},
  year = {2026},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Sudharsanselvaraj/Token-Print}}
}