MoonshotAI GitHub avatar

Kimi-K3

MoonshotAI

Kimi K3 is an open-weight, 2.8T-parameter native multimodal agentic model with a 1M-token context window, designed for frontier coding, knowledge work, and reasoning tasks.

Stars

3,348

7-day growth

No data

Forks

269

Open issues

10

License

NOASSERTION

Last updated

2026-07-28

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It is the world’s first open 3T-class model, featuring a novel Kimi Delta Attention architecture and MoE sparsity that delivers a 2.5× scaling efficiency improvement over its predecessor, while achieving top-tier scores on coding, agentic, and vision benchmarks.

Who it is for

  • AI researchers exploring large-scale MoE architectures
  • Software engineers building long-horizon agentic systems
  • Enterprise teams needing multimodal, long-context AI for knowledge work
  • Developers seeking openly available frontier-level model weights

Use cases

  • Long-horizon coding such as GPU kernel optimization and compiler development
  • Agentic knowledge work including deep research with interactive dashboards and visualizations
  • Vision-in-the-loop tasks like game development, CAD, and chip design
  • Multi-tool orchestration and terminal-based automation

Strengths

  • Open release of full 2.8B-parameter weights under a permissive license
  • Native multimodality (text and image) with a 1-million-token context window
  • State-of-the-art performance on agentic benchmarks (e.g., BrowseComp, MCPMark-Verified, DeepSearchQA)
  • Scaling efficiency improved 2.5× via KDA and Stable LatentMoE with 896 experts

Considerations

  • Model size (2.8T total parameters) demands substantial computational resources for inference
  • Explicit support only for text and image inputs (video understanding claimed but not fully detailed in the table)
  • License terms may impose restrictions beyond typical open-source licenses

README quick start

📰  Tech Blog |     📄  Full Report

1. Model Introduction

Kimi K3 is an open-weight, native multimodal agentic model and our most capable model to date. It is a 2.8T-parameter model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

Key Features

  • New Architecture: Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), and scales up MoE sparsity with a Stable LatentMoE framework that activates 16 out of 896 experts — yielding an approximate 2.5× improvement in overall scaling efficiency over Kimi K2.
  • Long-Horizon Coding: Operating with minimal human oversight, Kimi K3 sustains long engineering sessions, navigates massive repositories, and orchestrates terminal tools — from GPU kernel optimization and compiler development to vision-in-the-loop game dev, CAD, and even chip design.
  • Agentic Knowledge Work: Kimi K3 advances end-to-end knowledge work, producing deep research with interactive visualizations, widgets and dashboards, and motion design and video editing, powered by its native multimodal architecture.
  • Native Multimodality & Long Context: Kimi K3 understands text, images, and video within the same model, and supports a 1-million-token context window.
  • Open Frontier Weights: We release the full Kimi K3 model weights under the Kimi K3 License, making frontier intelligence openly available for research, deployment, and further innovation.

2. Model Summary

Architecture Mixture-of-Experts (MoE)

Total Parameters 2.8T

Activated Parameters 104B

Number of Layers 93

Number of Dense Layers 1

Attention-Layer Composition 69 KDA + 24 Gated MLA

Attention Hidden Dimension 7168

Number of Attention Heads 96

Latent MoE Dimension 3584

MoE Hidden Dimension (per Expert) 3072

Number of Experts 896

Selected Experts per Token 16

Number of Shared Experts 2

Vocabulary Size 160K

Context Length 1048576

Attention Mechanism KDA & Gated MLA

Activation Function SiTU-GLU

Vision Encoder MoonViT-V2

Parameters of Vision Encoder 401M

Quantization MXFP4 weights / MXFP8 activations(quantization-aware training)

Modalit

Description

Open Frontier Intelligence

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
jamesob
Featured
jamesob GitHub avatar

local-llm

A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.

AI & Machine LearningLarge Language Models
1,660
mshumer
Featured
mshumer GitHub avatar

Claude-of-Duty

A browser-based first-person shooter built entirely with procedural generation and orchestrated AI agents, featuring 55k lines of Three.js/WebGL2 code and no art assets.

AI & Machine LearningAI Agents
1,234