kimi-k3-in-c

kimi-k3-in-c

A 2.78-trillion-parameter model. One CPU. 8 GB of RAM.

Kimi K3 inference in portable C99.No BLAS. No framework. No GPU.

2.78Tparameters 1.56 TBcheckpoint on disk 8.24 GBpeak RSS, measured 176 KBthe whole engine 0GPUs

$ ./bin/k3 ~/k3model --trunk ~/k3trunk --preset laptop \
           --tok ~/k3model --prompt "The capital of France is" --gen 8 --incremental

--- generated text ---
 Paris.",
+            "The Eiffel
----------------------
8 tokens in 261.5 s, 32.69 s/token average
PEAK RSS for the whole run: 8.24 GB

Slow, and answering correctly, in 8.24 GB, from a checkpoint of 1.56 TB. This is a base model, so what follows " Paris." is a continuation rather than a reply; there is no chat template. Give it more memory and the answer does not change, only the clock:

$ ./bin/k3 ~/k3model --trunk ~/k3trunk --preset server \
           --tok ~/k3model --prompt "def fibonacci(n):" --gen 28 --incremental

--- generated text ---
    if n  [prompt] [memory] [generation] [diagnostics]

`` is the directory holding the .safetensors shards. It is required for any run, but --help, --version and --list-presets work without it:

./bin/k3 --help
./bin/k3 --version
./bin/k3 --list-presets

Prompt options

Exactly one of these is required. Passing none, or more than one, is a usage error (exit 2).

# text in
./bin/k3 ~/k3model --tok ~/k3model --prompt "The capital of France is" ...

# text in, from a file. Use this for CJK, emoji, accents
printf 'La capitale de la France est' > /tmp/p.txt
./bin/k3 ~/k3model --tok ~/k3model --prompt-file /tmp/p.txt ...

# ids in, ids out, no tokenizer needed
./bin/k3 ~/k3model --ids 1008,10484,318,15383,387 ...

Memory options