Blaizzy
Featured
nativ
Nativ is a native macOS app that lets you run AI models locally on Apple silicon, offering chat, model management, performance analytics, and an OpenAI/Anthropic-compatible API server.
inference-school
Inference School is a problem-based course that guides learners through building a small language-model inference engine for Apple Silicon, with 47 exercises covering everything from dot products to a capstone inference engine.
Stars
180
7-day growth
No data
Forks
14
Open issues
0
License
Apache-2.0
Last updated
2026-07-17
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It uniquely combines CPU and Metal GPU implementations with a performance-measurement-first approach, requiring learners to predict, measure, and integrate each operator into a growing engine rather than just passing tests.
Who it is for
- Apple Silicon developers learning Metal and GPU programming
- Machine learning engineers wanting to understand inference internals
- Students of performance optimization and systems programming
- Hobbyists building custom inference solutions on macOS
Use cases
- Self-paced study of transformer inference from first principles
- Hands-on practice with Metal GPU kernel design and optimization
- Understanding and profiling KV-cache, quantization, and batching techniques
- Building a reference educational inference engine for experimentation
Strengths
- Complete 47-problem progressive curriculum with fully runnable exercises
- Every operator has both a Swift CPU oracle and a Metal GPU implementation to compare
- Companion book (545-page PDF and EPUB) with diagrams and chapter-end solutions
- Performance prediction and measurement integrated into every problem
Considerations
- Requires an Apple Silicon Mac with macOS 15 or newer, limiting platform adoption
- Does not include a pretrained model or complete Metal generation for the final capstone
- Command-line checks inherit terminal permissions, requiring security awareness
README quick start
Inference School
Inference School is a problem-based course in which every exercise contributes to one small language-model inference engine for Apple Silicon. The problems are not standalone interview puzzles. A dot product becomes GEMV, GEMV becomes Q/K/V projections, those projections become attention, and the same path eventually loads a model and generates tokens.
Each numerical operator has two required paths where a GPU mapping is part of the lesson:
- A readable Swift CPU implementation that acts as the correctness oracle.
- A Metal implementation whose memory traffic, dispatch shape, and numerical behavior you can explain and measure.
The finish line is not merely "the tests pass." For each problem, you predict performance, run an experiment, inspect the result, and integrate the operator into the growing engine.
How you use it
Inference School Studio is the primary learning environment. It is a native macOS app with a searchable lesson reader, editable Swift and Metal source files, out-of-process checks, persistent completion lists, diagrams, math rendering, and text scaling from 80% to 200%. The command line exposes the same lessons, learner files, judges, and benchmarks.
From the repository root, package and open the Studio:
scripts/package-studio.sh debug
open "dist/Inference School Studio.app"
The packaging script creates an ad hoc signed, sandboxed app for local use. It does not produce a Developer ID signature or a notarized distribution.
It opens at the reader-only orientation lesson 000. Read
Start Here for setup, local-execution boundaries, and the
intended learning loop. Problems 001 through 047 are runnable, including the
final profiling and systems module.
The packaged Studio is signed with App Sandbox. The first runnable lesson asks you to choose a dedicated build folder through the macOS folder picker. Editable sources, compiler output, and generated learner executables stay under that folder. The Studio host has the client entitlement required by WebKit on supported macOS releases; its lesson, diagram, and editor assets are bundled, and built-in checks do not upload learner code. Command-line checks are separate: they run with the permissions of your terminal session.
Companion book
The complete course is available as a 545-page PDF and a reflowable EPUB with native diagra
Description
A hands-on Swift and Metal course for building LLM inference from first principles on Apple silicon, with 48 guided lessons, runnable exercises, a native macOS Studio, and a complete companion book.
Related repositories
Similar projects matched by category, topics, and programming language.
uzairansaruzi
Featured
hermex
Hermex is a native SwiftUI iPhone app that lets you control a self-hosted Hermes AI agent directly from your phone, with no subscriptions, tracking, or third-party relay.
simonw
Featured
speech-analyzer-cli
A macOS command-line tool that uses Apple's on-device speech APIs to transcribe prerecorded audio into plain text, JSON, JSONL, SRT, or WebVTT.