deerwork-ai
Featured
deer-workflow
An open-source Dynamic Workflow runtime that combines deterministic TypeScript orchestration with replaceable Agent runtimes.
deltafin
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.
Stars
204
7-day growth
No data
Forks
15
Open issues
0
License
NOASSERTION
Last updated
2026-07-29
Why it is worth attention
It demonstrates that a model far larger than a laptop’s RAM can be run locally by exploiting the sparsity of MoE layers, combining careful I/O, fused kernels, and double-buffering to achieve feasible (if slow) inference on consumer hardware.
Who it is for
- Researchers studying large MoE model inference
- Machine learning engineers exploring on-device inference
- Hobbyists interested in running state-of-the-art models on a Mac
- Students and practitioners learning about quantization and streaming techniques
Use cases
- Running K3's natural language generation on a Mac for experimentation
- Studying router behavior via logged traces (router_trace.jsonl)
- Serving a compatible OpenAI API endpoint for integration with chat interfaces
- Testing streaming-inference techniques and I/O optimization strategies
Strengths
- Exact, reproducible greedy decoding matching the reference model
- Works on a 64 GB MacBook – a machine far below the model's memory footprint
- Provides both full-download and streaming modes, with easy upgrade path
- Open-source (MIT) with transparent performance measurements and failure documentation
Considerations
- Very slow – about 16 seconds per token on M1 Max (3.75 tokens/minute)
- Requires ~1.7 TB disk for full local install; streaming mode introduces network latency
- Greedy-only decoding (temperature/top_p ignored); single concurrent request
README quick start
Usage
Description
Run Kimi K3, a 2.8T-parameter Mixture-of-Experts LLM, on a single Apple Silicon Mac. Streams MXFP4 experts on demand over HTTP into a local disk cache — fused NEON kernels, Metal/MPS compute, exact reproducible decoding, and an OpenAI-compatible API server for local chat and coding agents.
Related repositories
Similar projects matched by category, topics, and programming language.
7-e1even
Featured
learn-agent
A collection of notes on building coding agents, derived from the production development of the desktop agent Reina, with each mechanism simplified into a zero-dependency, single-file Node.js demo.
slvDev
Featured
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.