它提供了一个完整、可复现且透明的管道，用于构建小规模领域特定语言模型，每一步都结合了实际设计决策和真实测量，让 LLM 预训练变得易于学习。
Train a small GPT for earthquake science — the entire LLM lifecycle, from a blank folder to a talking model, explained block by block.
Crawl → Clean → Tokenize → Model → Train → Infer, on 2× NVIDIA A30 (48 GB).
Six free data sources → crawl → clean/dedup → 16k BPE → 113M GQA+RoPE decoder → 2-GPU DDP training → streaming inference. Each stage is a section below.
nanoGPT-Seis is a teaching repository. It is not trying to be a great earthquake model — it is trying to make every stage of pretraining a language model legible: where the data comes from, how it is cleaned and deduplicated, how a tokenizer is built, why the Transformer looks the way it does, how it is trained across two GPUs, and how it is served. Every design decision is explained, and every number (perplexity, VRAM, tokens) is one we actually measured on this hardware.
The corpus mixes earthquake / seismology text (open-access papers via Crossref+Unpaywall, arXiv/EarthArXiv preprints, the "Earthquake Insights" Substack) with general text (Wikipedia + FineWeb-Edu) for plain-language fluency — about 24% domain / 76% general. A focused corpus lets a ~100M-param model become genuinely fluent on a single node, so you can see the whole loop close in a day, not a month. (Why the general mix? See §1.)
Status: the pretraining lifecycle is complete — crawl through inference.
| value | |
|---|---|
| Corpus | 533,248 docs · 485.7M words · 822.7M training tokens (≈2.4:1 general:domain) |
| Model | 113M params — decoder-only, GQA + RoPE + RMSNorm + SwiGLU |
| Hardware | 2× NVIDIA A30 (24 GB each), bf16, DDP — also runs on a single RTX 3090/4090, or 12–16 GB with a smaller batch ([§7.5](#75-running-on-on |
根据分类、Topic 和编程语言匹配的相似项目。
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.
