Why it is worth attention

It is one of the few projects to get llama.cpp running natively on a Nintendo Switch via a custom patch, proving that even a Tegra X1 can handle offline quantized LLMs in a constrained homebrew environment.

Who it is for

Nintendo Switch homebrew users
Developers interested in llama.cpp ports to unusual platforms
Privacy-minded tinkerers who want offline LLM chat
Hobbyists exploring low-power edge LLM inference

Use cases

Running small Q4 GGUF models on a Switch without internet
Experimenting with llama.cpp CPU inference on Tegra X1 hardware
Keeping private multi-turn chat logs on an SD card
Building and extending a homebrew LLM console app with devkitPro

Strengths

Works fully offline and CPU-only, needing no cloud or GPU
Accepts any .gguf model dropped into a folder and persists chat histories as plain text
Ships a llama.cpp Switch/newlib patch and toolchain file for reproducible builds
Simple console UI with model picker and multi-chat management

Considerations

No GPU acceleration and only small Q4 models are practical due to 4 GB of RAM
Expect roughly 1–3 tokens/second, so it is a fun toy rather than a production stack
Long conversations are re-fed every turn and can slow down or exceed the context window; UI is console-only

README quick start

Switch-Alpaka

Run local LLMs (GGUF models via llama.cpp) natively on a homebrew-enabled Nintendo Switch — no internet connection required. CPU-only inference, console-based UI, model picker, and persistent multi-chat history saved to the SD card. Disclaimer: This was Vibe Coded using Claude Sonnet 5

Features

Pick any .gguf model dropped into sdmc:/switch/llm/models/
Multiple saved chats per model, stored as plain text on the SD card
Multi-turn conversation context (not just single-shot Q&A)
Runs fully offline, CPU-only (Tegra X1, no GPU/CUDA support — see Background)

Requirements

A Switch with Atmosphère (or compatible) CFW and the homebrew launcher
devkitPro with devkitA64, switch-dev, and switch-tools installed
A quantized .gguf model (see Getting a model)

Building

1. Install devkitPro (if you haven't already)

wget https://apt.devkitpro.org/install-devkitpro-pacman
chmod +x ./install-devkitpro-pacman
sudo ./install-devkitpro-pacman
sudo dkp-pacman -S devkitA64 switch-dev switch-tools

Make sure your environment has:

export DEVKITPRO=/opt/devkitpro
export DEVKITA64=${DEVKITPRO}/devkitA64
export PATH=${DEVKITPRO}/tools/bin:${DEVKITA64}/bin:${PATH}

2. Clone and patch llama.cpp

llama.cpp doesn't support Horizon OS out of the box. This repo ships a patch that adds Switch/newlib compatibility (no dlfcn.h/mmap, missing posix_memalign/sysconf, forces the CPU backend, etc).

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git apply ../patches/llama-switch.patch

3. Cross-compile llama.cpp for the Switch

mkdir build-switch && cd build-switch
cmake .. -DCMAKE_TOOLCHAIN_FILE=../switch-toolchain.cmake \
  -DGGML_OPENMP=OFF \
  -DGGML_NATIVE=OFF \
  -DLLAMA_CURL=OFF \
  -DBUILD_SHARED_LIBS=OFF \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLAMA_BUILD_TOOLS=OFF \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_APP=OFF \
  -DGGML_BACKEND_DL=OFF \
  -DGGML_CPU=ON \
  -DLLAMA_BUILD_COMMON=OFF
make -j$(nproc)
cd ../..

The toolchain file (switch-toolchain.cmake) is included in this repo — copy it into your llama.cpp checkout before running cmake:

cp switch-toolchain.

Related repositories

Similar projects matched by category, topics, and programming language.

slvDev

Featured

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models

1,960

jamesob

Featured

local-llm

A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.

AI & Machine LearningLarge Language Models

1,660

makecindy

Featured

cindy

Cindy is an open-source AI agent that runs locally on your machine, integrates multiple AI harnesses and models, and provides memory, skills, and automation to perform real work in your projects and apps.

AI & Machine LearningLarge Language Models

958