MoonshotAI
精选
Kimi-K3
Kimi K3 is an open-weight, 2.8T-parameter native multimodal agentic model with a 1M-token context window, designed for frontier coding, knowledge work, and reasoning tasks.
AI 仓库情报摘要
FR-AI / ANALYSIS为什么值得关注
该仓库展示了将前沿规模的 MoE 模型移植到 MLX 的最全面工程，包括位精确的 MXFP4 转换、新型注意力残差架构、完全验证的视觉塔以及 REAP 专家剪枝——尽管模型太大无法在现有硬件上运行，但所有实现都有严格测试。
适合谁使用
- MLX 与 Apple Silicon 开发者
- 研究大型 MoE 或注意力残差架构的研究人员
- 将大规模模型移植到有限内存生态系统的工程师
- 关注专家剪枝与校准的从业者
典型使用场景
- 通过位精确的往返测试验证 MXFP4 量化精度
- 在真实 2.78T 模型上测试基于领域的 REAP 剪枝
- 研究 Kimi Delta 注意力与注意力残差在 MLX 中的行为
- 评估用于内存受限推理的双库专家量化方法
项目优势
- 端到端验证的位精确 MXFP4 转换（零误差）
- 全面的 39 项测试，涵盖架构、视觉塔精度和包装器
- 详细的移植挑战与设计决策文档
- REAP 剪枝配合领域定向校准，并有生成示例验证
使用前须知
- 没有一种压缩版本能在现有 Apple Silicon Mac 上运行（最小 870GB 对比最大 512GB 统一内存）
- 已发布的版本仅达到 0.14–0.20 tokens/sec，远非交互式使用
- 2-bit 和混合版本涉及二次量化；质量未经确认且可能较差
README 快速开始
kimi-k3-mlx
MLX (Apple Silicon) port of moonshotai/Kimi-K3 — Moonshot's 2.78T-total / 104B-active native-multimodal MoE, built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with a 1M-token context.
kimi_k3.py is a single, stock-mlx-lm-compatible model definition for the text
tower; kimi_k3_vision.py is the vision tower and projector; kimi_k3_vl/ is
the mlx-vlm wrapper that joins them.
scripts/convert.py is a streaming converter — mlx_lm convert cannot be used
here, because it materialises the whole model and K3 is 5.6 TB in bf16.
Read this before planning a deployment. Nothing in this repo runs on a single Mac. The smallest tier is ~870 GB against a 512 GB ceiling on the largest Apple Silicon machine that exists. See Reality check.
Architecture
| Feature | Detail |
|---|---|
| MoE | 896 routed experts, top-16, moe_intermediate_size=3072; sigmoid router + aux-loss-free e_score_correction_bias; 2 shared experts on every token |
| Stable LatentMoE | routed experts run in a 3584-d latent, not the 7168-d residual: down_proj → experts → RMSNorm → up_proj |
| Attention | 93 layers: 69 KDA + 24 gated MLA, 96 heads, head_dim=128 |
| KDA | short conv (k=4) on q/k/v, low-rank forget gate (f_a/f_b), full-rank output gate, gate_lower_bound=-5.0 |
| MLA | q-LoRA rank 1536, kv-LoRA rank 512, NoPE (no rotary at all), sigmoid output gate (g_proj) |
| AttnRes | every 12 layers pushes the residual onto a stack; each sub-layer input is a softmax mixture over that stack, scored by a rank-1 [1, 7168] direction |
| Activation | SiTU-GLU: β·tanh(g/β)·σ(g) · β_lin·tanh(u/β_lin), β=4, β_lin=25 |
| Dense layer | layer 0 only (first_k_dense_replace=1, intermediate_size=33792) |
| Vocab | 163,840 · tiktoken · tie_word_embeddings=False |
| Source format | routed experts ship as MXFP4 (weight_packed u8 + e8m0 weight_scale, group 32); everything else bf16 |
Parameter budget — 97.94% of the model is routed experts:
| component | params | share |
|---|---|---|
| routed experts (MXFP4 in source) | 2722.7 B | 97.94% |
| KDA attention ×69 | 30.6 B | 1.10% |
| shared experts | 12.2 B | 0.44% |
| MLA attention ×24 | 5.6 B | 0.20% |
| latent-MoE up/down | 4.7 B | 0.17% |
| embed + lm_head |
项目描述
MLX port of moonshotai/Kimi-K3 (2.78T multimodal MoE): streaming converter, REAP expert pruning, and per-language expert-overlap analysis
相关仓库与替代方案
根据分类、Topic 和编程语言匹配的相似项目。
jamesob
精选
local-llm
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.
Blaizzy
精选
nativ
Nativ is a native macOS app that lets you run AI models locally on Apple silicon, offering chat, model management, performance analytics, and an OpenAI/Anthropic-compatible API server.