It achieves 113.83 tok/s on a single L40S under a 32GB cap, uses a protocol adapter that detects and translates multiple client dialects transparently, and provides transparent benchmarked performance with detailed kernel optimization.
Sketch is illustrative. This repository serves Kimi-Linear-48B, measured on an NVIDIA L40S inside a hard 32 GiB cap. The full Kimi K3 is 2.78T parameters and runs on an 8x B300 node, linked below.
TL;DR. Run Kimi on your own consumer GPU, not someone's API. A single 32 GB card (RTX 5090, or any datacenter card) serves Kimi-Linear-48B at 113.83 tok/s, up 3.18x from fused INT4 kernels, and Claude Code, Codex, Cline, Aider and opencode all connect to it unchanged.
Need the full Kimi K3 in production? 2.8 trillion parameters will not fit a workstation. If you want it running inside your own network instead of behind a closed-source API, RunInfra ships it as a deployable package with a pinned vLLM build, a benchmark receipt and weight verification: Kimi K3 on 8x B300 →
Point any coding agent at a local Kimi model and it works: k3 translates between the agent's protocol and the model's.
flowchart LR
CC["Claude Code"] -- "Anthropic Messages" --> K3
CX["Codex"] -- "OpenAI Responses" --> K3
OT["Aider, Cline, opencode"] -- "OpenAI Chat" --> K3
K3{{"k3detects the client per requesttranslates all three dialectstool calls: hermes, json, kimi,kimi_k3, pythonic, passthroughrestores reasoning across turns"}}
K3 -- "OpenAI Chat" --> EN["local engineKimi-Linear-48Bselective INT4fused decode kernels"]
classDef client fill:#0d1117,stroke:#30363d,color:#c9d1d9
classDef bridge fill:#1f6feb,stroke:#388bfd,color:#ffffff
classDef engine fill:#238636,stroke:#2ea043,color:#ffffff
class CC,CX,OT client
class K3 bridge
class EN engine
Local model servers usually speak OpenAI Chat Completions. Claude Code speaks Anthropic Messages. Codex speaks OpenAI Responses. Point the wrong client at the wrong endpoint and the request fails. k3 sits between them, detects the caller on each request, and translates the request and response.
The protocol landscape has changed. Current releases of vLLM, llama.cpp, and [Ollama](https://docs.o
Optimized local serving engine for Kimi-Linear-48B: INT4 quantizer, fused decode kernels for a measured 3.18x, and an OpenAI-compatible server. Ships with k3, a bridge that detects the client per request so Claude Code, Codex, Cline, Aider and opencode all work unchanged.
Similar projects matched by category, topics, and programming language.
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.
Cindy is an open-source AI agent that runs locally on your machine, integrates multiple AI harnesses and models, and provides memory, skills, and automation to perform real work in your projects and apps.
OpenClaude is an open-source CLI coding agent that runs on any platform and supports a wide range of LLM providers, offering the same tools and workflows as Claude Code.