MoonshotAI/Kimi-K3 - 仓库详情

AI 仓库情报摘要

FR-AI / ANALYSIS

为什么值得关注

它是全球首个开放的 3T 级模型，采用创新的 Kimi Delta Attention 架构和 MoE 稀疏性，相比前代具有 2.5 倍的扩展效率提升，并在编程、智能体和视觉基准测试中取得顶尖成绩。

适合谁使用

研究大规模 MoE 架构的 AI 研究者
构建长周期智能体系统的软件工程师
需要多模态、长上下文 AI 的企业知识工作团队
寻求开放前沿级模型权重的开发者

典型使用场景

GPU 内核优化和编译器开发等长周期编程任务
深度研究、交互式仪表盘和可视化等智能体知识工作
游戏开发、CAD 和芯片设计等视觉反馈循环应用
多工具编排和终端自动化

项目优势

完整开放 2.8T 参数权重，许可协议相对宽松
原生多模态（文本与图像）并支持 100 万 token 上下文
在智能体基准（如 BrowseComp、MCPMark-Verified、DeepSearchQA）上达到顶级水平
通过 KDA 和 Stable LatentMoE（896 专家）实现 2.5 倍扩展效率提升

使用前须知

2.8T 总参数量导致推理需要大量计算资源
表格中仅明确支持文本和图像输入（视频理解虽声称但未充分展示）
许可协议可能带有超出常见开源许可的限制

README 快速开始

📰 Tech Blog | 📄 Full Report

1. Model Introduction

Kimi K3 is an open-weight, native multimodal agentic model and our most capable model to date. It is a 2.8T-parameter model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

Key Features

New Architecture: Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), and scales up MoE sparsity with a Stable LatentMoE framework that activates 16 out of 896 experts — yielding an approximate 2.5× improvement in overall scaling efficiency over Kimi K2.
Long-Horizon Coding: Operating with minimal human oversight, Kimi K3 sustains long engineering sessions, navigates massive repositories, and orchestrates terminal tools — from GPU kernel optimization and compiler development to vision-in-the-loop game dev, CAD, and even chip design.
Agentic Knowledge Work: Kimi K3 advances end-to-end knowledge work, producing deep research with interactive visualizations, widgets and dashboards, and motion design and video editing, powered by its native multimodal architecture.
Native Multimodality & Long Context: Kimi K3 understands text, images, and video within the same model, and supports a 1-million-token context window.
Open Frontier Weights: We release the full Kimi K3 model weights under the Kimi K3 License, making frontier intelligence openly available for research, deployment, and further innovation.

2. Model Summary

Architecture Mixture-of-Experts (MoE)

Total Parameters 2.8T

Activated Parameters 104B

Number of Layers 93

Number of Dense Layers 1

Attention-Layer Composition 69 KDA + 24 Gated MLA

Attention Hidden Dimension 7168

Number of Attention Heads 96

Latent MoE Dimension 3584

MoE Hidden Dimension (per Expert) 3072

Number of Experts 896

Selected Experts per Token 16

Number of Shared Experts 2

Vocabulary Size 160K

Context Length 1048576

Attention Mechanism KDA & Gated MLA

Activation Function SiTU-GLU

Vision Encoder MoonViT-V2

Parameters of Vision Encoder 401M

Quantization MXFP4 weights / MXFP8 activations(quantization-aware training)

Modalit

项目描述

Open Frontier Intelligence

Kimi-K3