API Reference

Reference

Base URL (dev): http://localhost:8000. All responses are JSON. CORS is restricted to http://localhost:3000.

GET /health

Liveness check — whether the backend is running and the model is loaded.

json
{ "status": "ok", "model_loaded": true }

GET /model-info

Real model metadata from the loaded PyTorch model.

json
{
  "model": "Qwen/Qwen2.5-0.5B-Instruct",
  "device": "mps",
  "num_layers": 24,
  "num_heads": 14,
  "hidden_size": 896,
  "attn_implementation": "eager",
  "max_tokens": 40,
  "ready": true
}

GET /architecture

Real architecture metadata and tensor list from named_parameters() and config. No forward pass required.

json
{
  "source": "model",
  "model": "Qwen/Qwen2.5-0.5B-Instruct",
  "metadata": {
    "architecture": "qwen2",
    "total_params": 494032768,
    "num_layers": 24,
    "num_heads": 14,
    "num_kv_heads": 2,
    "head_dim": 64,
    "ffn_size": 4864,
    "vocab_size": 151936,
    "context_length": 32768,
    "rope_theta": 1000000.0
  },
  "tensor_count": 290,
  "tensors": [
    {
      "name": "model.embed_tokens.weight",
      "shape": [151936, 896],
      "dtype": "float32",
      "n_params": 136134656
    }
  ]
}

POST /analyze

Runs a real forward pass and returns attention weights, token embeddings, PCA projections, and the logit lens for the given sentence. Capped at approximately 40 tokens.

request body
{ "sentence": "The cat sat on the mat." }
response (abbreviated)
{
  "sentence": "The cat sat on the mat.",
  "num_layers": 24,
  "num_heads": 14,
  "tokens": [
    { "index": 0, "text": "The", "id": 785, "is_special": false }
  ],
  "attention": [ /* [layer][head][from][to] — shape [24, 14, 7, 7] */ ],
  "embeddings_3d": [ [x, y, z] ],
  "logit_lens": [ /* [layer][position] -> [{text, id, prob}] */ ]
}

WS /ws/generate

Real streamed greedy generation over a WebSocket connection.

client → server
{
  "prompt": "Name one primary color.",
  "max_new_tokens": 40,
  "top_k": 10,
  "trace": true
}

The server sends three frame types:

frame: meta (once)
{
  "type": "meta",
  "model": "Qwen/Qwen2.5-0.5B-Instruct",
  "num_layers": 24,
  "prompt_len": 36,
  "uses_kv_cache": true,
  "op_catalog": [
    {
      "index": 0,
      "op_key": "embedding",
      "label": "Token Embedding",
      "layer": null,
      "param_count": 136134656,
      "cumulative_params": 136134656,
      "in_dim": 896,
      "out_dim": 151936
    }
    // 243 ops total
  ]
}
frame: token (per generated token)
{
  "type": "token",
  "step": 0,
  "chosen": { "id": 6893, "text": "Red", "logprob": -0.04 },
  "topk": [
    { "id": 6893, "text": "Red", "logit": 18.4, "prob": 0.957 }
  ],
  "phase": "prefill",
  "n_positions": 39,
  "cache_len": 0,
  "eos": false
}
frame: done
{
  "type": "done",
  "generated_text": "Red",
  "total_steps": 2
}

POST /ablate/analyze

Zeroes selected heads or layers via forward hooks and runs a real forward pass. Response has the same shape as /analyze — diff the two logit-lens tables to measure the effect of the ablation.

Note

Ablation operates on raw architectural components — any head, any block, on whatever model is loaded. No pre-trained sparse autoencoder or transcoder required.

GET /trace · POST /trace/replay

Record and replay. When record_trace: true is set on a generation request, the full WebSocket stream is saved. GET /trace returns the last recorded run as a downloadable .tokenprint.json. POST /trace/replay accepts an uploaded trace and replays it back.

Important

Traces are versioned (trace_version: 1). Replay code rejects unknown versions rather than silently mis-reading them.