A distributed inference system for running 30-70B LLMs on everyday home devices that achieves up to 15x speedup over llama.cpp while maintaining memory pressure below 10%.

Stars

28

7-day growth

No data

Forks

No data

Open issues

No data

License

No data

Last updated

2026-06-30

Description

[Official] prima.cpp: Fast 30-70B LLM inference on heterogeneous and everyday home devices

Related repositories

Similar projects matched by category, topics, and programming language.

ammaarreshi
ammaarreshi GitHub avatar

Generals-Mac-iOS-iPad

This repository ports Command & Conquer Generals: Zero Hour to run natively on Apple Silicon Macs, iPhones, and iPads without emulation, adding touch controls and modern rendering.

C++
1,456
MSNightmare
MSNightmare GitHub avatar

LegacyHive

A proof-of-concept for a Windows user profile service elevation of privilege vulnerability, capable of loading arbitrary hives on all supported systems up to July 2026 patch.

C++
182
kigner
kigner GitHub avatar

audio.cpp-webui

A native C++ audio inference framework that runs multiple audio models significantly faster than Python while supporting a wide range of tasks like TTS, ASR, and voice cloning.

C++
144