llama.cpp — Fermion Research fork (fermion-fv5)

This is Fermion Research's fork of ggml-org/llama.cpp. It adds two GGML weight types so llama.cpp can load and run Neutrino GGUF packs — Fermion's ternary-family models — on the CPU and CUDA backends:

Everything else is stock llama.cpp at the pinned base commit — the model graph is untouched (Neutrino models are stock Qwen3 geometry, which mainline already implements). The upstream README is preserved as README_LLAMACPP.md.

GGUF is the compatibility door. Fermion's native runtime and its fused kernels remain the fast door; this fork optimizes for correctness parity with the deployed container function, not for peak speed.

Quickstart

You do not convert anything. Download a ready-made Neutrino GGUF pack (e.g. neutrino-8b-fv5.gguf, ~4.1 GB, or the 0.6B cut), verify its SHA256SUMS, then build this fork and point it at the file.

# CPU only
cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF
cmake --build build -j --target llama-server llama-bench llama-completion

# CPU + CUDA (full offload supported)
cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-bench llama-completion

Run an OpenAI-compatible server, fully offloaded:

./build/bin/llama-server -m neutrino-8b-fv5.gguf -ngl 99 -c 4096 --host 127.0.0.1 --port 8080

Or a quick greedy completion / benchmark:

./build/bin/llama-completion -m neutrino-8b-fv5.gguf -ngl 99 -p "The capital of France is" -n 64 --temp 0 -no-cnv
./build/bin/llama-bench -m neutrino-8b-fv5.gguf -ngl 99 -p 512 -n 128

Stock upstream llama.cpp will reject these packs (FV5B = type id 44 is out of range there) — you need this fork until the types are upstreamed.

Correctness

The fork ships a gate tool, tools/fermion-greedy (llama-fermion-greedy), that certifies greedy token identity (free-running and teacher-forced, with per-flip logit-margin forensics, F32 KV cache):