bcallender GitHub avatar

agent-context-workshop

bcallender

A workshop that builds a code knowledge graph for coding agents and evaluates its effectiveness against traditional grep-based context.

Stars

16

7-day growth

No data

Forks

8

Open issues

0

License

Apache-2.0

Last updated

2026-06-29

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It provides an honest eval showing graph-based context improves determinism and efficiency over grep, with a novel metric of agent adoption.

Who it is for

  • Coding agent developers
  • AI/ML researchers focused on code understanding
  • Software engineers building developer tools
  • Rust developers interested in code analysis

Use cases

  • Improving coding agent accuracy on relationship questions
  • Reducing token costs in agent context
  • Enabling deterministic code queries
  • Evaluating context engineering strategies

Strengths

  • Replicated evaluation with 30 gold questions showing graph ties grep on accuracy but wins on determinism
  • ~3× fewer input tokens compared to grep
  • Novel adoption metric showing 72% agent usage with nudge vs 34% without
  • Open-source with Jupyter notebooks and cached eval data for reproducibility

Considerations

  • Example limited to a single Rust codebase (qdrant)
  • Full graph traversal requires Neo4j database
  • Results may not generalize to other languages or codebases

README quick start

Agent Context Workshop

A context layer for coding agents — built, then measured.

Most coding agents explore a codebase by grepping it. This workshop builds the alternative: a code knowledge graph an agent can traverse instead — then runs an honest eval to see whether it actually helps, and whether the agent uses it.

The worked example is qdrant (a real Rust vector database). We extract its structure into a Neo4j graph and give an agent graph tools (search, blast_radius, implementors, …). Then we pit a graph agent against a grep agent on 30 gold-verified questions.

The honest result: on single-shot accuracy they roughly tie (graph 21/30, grep 19/30). The graph wins on what matters in production — determinism (right on every run: graph 19/30 vs grep 11/30, and 12/15 vs 6/15 on relationship questions — grep re-derives the connections each run and flips), efficiency (~3× fewer input tokens), and the metric nobody reports: adoption (does the agent reach for the graph at all — 34% → 72% with the right nudge).

The two mental models

Three rungs of building the context layer (each notebook walks them):

RungWhatHowDeterminism
0 · syntacticsymbols, spanstree-sitterdeterministic
1 · resolutiontypes, impls, edgesrustdoc JSONdeterministic
2 · meaningintent, stabilityfenic + an LLMmodel-backed, cached

Three levels of context engineering (what the eval compares): L1 grep → L2 index + search → L3 graph traversal.

Quickstart

Prereqs: uv, Docker (optional — only for notebook 01's graph), and an OpenRouter key (optional — only for live LLM calls; the eval is cached).

uv sync                       # install (slim — one provider, via OpenRouter)
scripts/setup_data.sh         # hydrate the committed qdrant seed (~2MB) into data/
cp .env.example .env          # add OPENROUTER_API_KEY for live calls (optional)
uv run jupyter lab

Then — new here? jump to notebook 02 (the eval; no Docker, no key needed). The full set, in order:

  • notebooks/00_start_here.ipynb — a 30-second preflight: confirms your install + data are ready and routes you. Run it first.
  • notebooks/01_posting_list_context.ipynb — bu

Description

A context layer for coding agents — build a code knowledge graph from a real codebase, then measure whether the agent actually uses it. AI Engineer World's Fair 2026 workshop.

Related repositories

Similar projects matched by category, topics, and programming language.

l0ng-ai
Featured
l0ng-ai GitHub avatar

tty7

tty7 is a fast, GPU-accelerated terminal workbench with persistent sessions, built-in input tools, SSH support, and coding agent awareness.

Developer ToolsCLI & Terminal
359
steelbrain
Featured
steelbrain GitHub avatar

reims-vgpu

reims-vgpu is an experimental virtual GPU for macOS guests that uses QEMU to decode the guest's GPU command stream and execute it through Metal or Vulkan, leveraging the built-in AppleParavirtGPU driver without requiring custom kexts.

Desktop Apps
143
m-novotny
Featured
m-novotny GitHub avatar

memguard-rs

A Rust library that provides secure memory handling primitives including zeroization on drop, memory locking, constant-time comparison, and compile-time guarded regions, with zero dependencies and no_std support.

Embedded & IoTSecurity
131