lat-defuse-rag-repair

Mechanistic evaluation of SAE-based activation interventions for correcting distractor-induced errors in RAG.

Retrieval-augmented generation (RAG) feeds a language model retrieved documents next to a question. When one of those documents is misleading (a distractor), the model often abandons the correct answer it would otherwise give. Lat-Defuse is a published method that repairs this failure by adjusting the model's internal sparse-autoencoder features until the correct answer wins again. This repository builds a faithful adaptation of that method for Gemma-3-12B and then studies what the repair actually does inside the model.

The reference implementation is the official code of the ACL 2026 Findings paper "The Mechanics of Interference: Defusing Distractors in RAG via Sparse Autoencoder Interventions" by Christian Giannetti and coauthors: at commit eff2cdfe664bc38d9de9ec3fb047b25d471826df (MIT). We do not copy its code; we reproduce its behavior. See docs/upstream-notes.md.

Directory map

docs/               the science contract, upstream notes, dataset survey
configs/            experiment configuration files
scripts/            runnable entry points
data/               datasets and curated examples, never committed
results/            experiment outputs, never committed
reports/            tracked run reports, one per real run
src/latdefuse/      the package
  data/             example schema, prompts, curated-example loader
  model/            model loading, activation capture, activation replacement
  sae/              SAE loading and compatibility checks
  steering/         the Lat-Defuse optimizer and stopping rules
tests/              unit tests, runnable without a GPU

Quickstart

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pytest -q

How we work

• Pick an open issue by label. Issues labeled "good first issue" need no GPU and no downloads.
• One issue, one focused pull request. CI (ruff, mypy, pytest) must be green before review.
• Issues labeled "needs-gpu" require a machine with the real model; everything else runs locally on a tiny test model.
• Read docs/science-contract.md before touching anything under src/latdefuse/. It defines the numbers and rules the sci