Why it is worth attention

It makes the Jacobian Lens technique from interpretability research directly usable on quantized GGUF models (including Mixture-of-Experts) without requiring PyTorch or a full Hugging Face pipeline, and adds a live intervention system that can steer any OpenAI-compatible application.

Who it is for

Researchers studying mechanistic interpretability of LLMs
Developers building or modifying applications that use llama.cpp-served models
ML practitioners who want to debug or steer model outputs in real time
Hobbyists experimenting with local, quantized language models

Use cases

Visualizing which tokens dominate the residual stream at each layer and position
Steering or ablating specific concepts in generated text (e.g., correcting faulty two-hop reasoning)
Integrating live concept manipulation into a chat application via the OpenAI-compatible backend
Reproducing and extending the paper's findings on a variety of architectures and quantization levels

Strengths

Works on any GGUF model llama.cpp can load, including dense and MoE architectures
Readout weights are read directly from the model GGUF (dequantized via gguf-py) — no need for PyTorch or the original checkpoint
Fits and serves the lens with memory and file size scaling as O(n_layers·d_model²), independent of expert count
Supports live interventions (steer, swap, ablate) during both prompt processing and token generation, with a clean OpenAI API for external apps

Considerations

The default regression-fit lens is a correlational approximation; the paper's exact causal lens requires fitting with the original PyTorch code and converting the .pt file
Grid computation for the readout (positions × d_model × vocab) can be slow on very large models — using a layer stride or shorter prompts is recommended
The native activation server must be run instead of a stock llama-server to enable live interventions; attaching to an already-running llama-server is not supported

README quick start

jlens-gguf — the Jacobian Lens for llama.cpp / GGUF models

A GGUF-native implementation of the Jacobian Lens from Anthropic's Verbalizable Representations Form a Global Workspace in Language Models, plus an interactive visualizer & manipulator (in the spirit of Neuronpedia's J-Lens pages) that works against GGUF models served by llama.cpp — with live steering, concept swapping (patching), ablation, and steered generation.

                                  browser (web UI)
                                        │  JSON
                        ┌───────────────▼────────────────┐
                        │  bridge  (python, numpy)       │  lens math:
                        │  jlens_gguf.server             │  J_l transport, unembed,
                        │   • model GGUF → readout wts   │  top-k, ranks, J-lens vectors,
                        │   • lens GGUF  → J_l (+ bias)  │  steering / swap / ablate,
                        └───────┬────────────────────────┘  sparse decomposition
                                │  HTTP (binary activations, generic residual edits)
                        ┌───────▼────────────────────────┐
                        │  jlens-server  (C++, llama.cpp)│  the same GGUF file your
                        │   • captures l_out- per    │  llama-server mmaps — weights
                        │     layer/position             │  are shared page cache
                        │   • applies add / low-rank /   │
                        │     set edits mid-graph        │
                        │   • generation w/ edits active │
                        └────────────────────────────────┘

New here? Start with QUICKSTART.md — a friendly, task-oriented walkthrough. This README is the full reference.

The reference PyTorch implementation reads the residual stream with hooks and computes with torch tensors. Here, the lens lives in a GGUF file, readout weights (final norm + unembedding) are read directly from the model GGUF (dequantized with gguf-py, no torch), and residual-stream access + live interventions go through a small llama.cpp-based activation server that hooks the computation graph via ggml's scheduler eval callback.

Quickstart

# clone with the llama.cpp submodule
git clone --recursive  jlens-gguf   # or:

Description

Interactive Jacobian-Lens visualizer and live steerer for GGUF models on llama.cpp

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo

Featured

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents

2,390

slvDev

Featured

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models

1,960

littledivy

Featured

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning

1,482