7-e1even
Featured
learn-agent
A collection of notes on building coding agents, derived from the production development of the desktop agent Reina, with each mechanism simplified into a zero-dependency, single-file Node.js demo.
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It bridges the gap between mathematical intuition and code by requiring hand calculation before any automation, reinforced by three independent implementations (paper, Python, Go) that cross-check each other.
Who it is for
- Self-learners struggling with backpropagation theory
- Machine learning beginners with minimal math background
- Educators looking for interactive teaching materials
- Developers wanting to understand autograd and training internals
Use cases
- Learning backpropagation with concrete step-by-step arithmetic
- Verifying understanding by comparing hand calculations with assert-driven code
- Teaching neural network fundamentals in a low-barrier, reproducible way
- Experimenting with small models (bigram, attention) on real hardware or Colab
Strengths
- No required math beyond basic arithmetic; calculus concepts built from scratch
- Every lesson includes hand computation, code, and asserts that enforce correctness
- Three-language cross-check (hand, Python NumPy, Go) catches bugs and deepens understanding
- Lessons progress from single neuron to GPT-2 FLOPs and LoRA fine-tuning on a single 4090
Considerations
- Some lessons (0009, 0010) require a GPU, limiting free-tier experimentation
- Focuses on tiny, pedagogical examples; does not cover production-scale training
- The Go library is supplementary and may not be needed for the core learning path
README quick start
soroban
[!NOTE] The GPU lessons in this repo are developed and tested on a real RTX 4090 machine generously provided by longkt90. None of the hardware measurements would exist without it. Thank you.
Soroban (算盤) is the Japanese abacus, the tool for doing real arithmetic by hand. That is the whole idea of this repo: learn to train neural networks by computing the training runs yourself, on paper, with a four-function calculator, and only then letting the machines confirm your numbers.
I started this because every explanation of backpropagation I read either hid the arithmetic behind matrix notation or hid it behind a framework. Neither ever made it stick for me. What made it stick was sitting down and computing three steps of gradient descent by hand, then writing code whose only job was to agree with my paper. So that is the format of every lesson here.
Every lesson follows the same discipline. You predict what will happen, you compute a small training run entirely by hand, then code reproduces your arithmetic with assert statements standing guard over every number, and finally you break something on purpose and study the wreckage. If the asserts pass, the machine has agreed with your paper. If they fail, you get to find out who is wrong, and learning to find out who is wrong is the actual curriculum.
There is no required background beyond arithmetic. The one calculus idea we need (a derivative is a slope you can measure by nudging) is built from a numerical experiment inside lesson 0001, not imported from a course you were supposed to have taken. And when notation gets in the way, the maths shelf has one plain-language page per symbol and idea, each worked out on the same numbers the lessons use, so the lessons can stay focused on training while the shelf carries the reference material.
The lessons
| # | Lesson | You compute by hand | Runs on | Status |
|---|---|---|---|---|
| 0001 | one neuron learns a line | 3 full gradient descent steps, every multiplication shown | any CPU | done |
| 0002 | a hidden layer learns a V | a full backward pass through two layers, all seven gradients | any CPU | done |
| 0003 | [ice, water, steam](lessons/0003-classific |
Description
Learn to train neural networks by hand: paper arithmetic, asserted code, and a from-scratch Go autograd, one lesson at a time
Related repositories
Similar projects matched by category, topics, and programming language.
slvDev
Featured
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
jamesob
Featured
local-llm
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.