A publicly archived dataset of 638 LLM inference benchmarks from a 4x RTX 3090 home rig, with full per-run metadata including exact launch commands and queryable via SQLite, JSONL, and Markdown catalog.
11
No data
1
0
No data
2026-07-28
It provides an unusually detailed and reproducible benchmark archive that preserves model checkpoint, quantization, KV dtype, TP layout, engine build, and launch command for every run, enabling accurate cross-comparison and analysis that typical tok/s numbers alone cannot offer.
638 local inference measurements from one 96 GB home rig. The exact launch command is attached to 637 of them.
Most benchmark posts give you a model name and one tok/s number. I kept
losing the rest: which checkpoint, which quant, which KV dtype, which TP layout,
which engine build, and whether the number meant single-stream decode or
aggregate throughput.
This repository is the database I wanted. You can inspect one result in the Markdown catalog, query all of them in SQLite, or take the JSONL and run your own analysis.
| Open | What is there |
|---|---|
| Models | All 14 model families and their runs |
| Benchmark campaigns | Matched experiments and test matrices |
| Engines | vLLM, llama.cpp, and the experimental exllamav3 slice |
| Query guide | Ready-to-run SQLite, JSONL, and Python examples |
| Methodology | Metric definitions, comparison rules, and limitations |
| Hardware | The 4x RTX 3090 rig, topology, power limits, and software versions |
| Dataset files | JSONL, SQLite, snapshot metadata, and schema notes |
The archive currently has 638 measurements from 20 benchmark campaigns across 14 model families. Most runs use vLLM. The llama.cpp set is smaller, and the 38 exllamav3 rows came from a short experiment rather than broad engine coverage.
This is a benchmark archive, not a leaderboard. The harness and the variable I was testing changed between campaigns, so some rows should not be compared directly. Each row keeps its workload, topology, power limit, engine version, source references, and launch command when available. 637 of 638 measurements retain an exact argv or command.
Every result keeps the context needed to understand or reproduce it:
| Recorded with the number | Examples |
|---|---|
| Model artifact | checkpoint, quantization recipe, source revision |
| Runtime | engine, version, attention backend, KV-cache dtype |
| GPU layout | TP, PP, DP, instance count, power limit, P2P state |
| Workload | prompt length, output length, concurrency, context limit |
| Metric meaning | single-stream wall rate, decode rate, aggregate output, TTFT |
| Receipt |
Local LLM speed measurements from a 4x RTX 3090 rig, with exact launch commands and per-run provenance.
Similar projects matched by category, topics, and programming language.
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.