uzairansaruzi
精选
hermex
Hermex is a native SwiftUI iPhone app that lets you control a self-hosted Hermes AI agent directly from your phone, with no subscriptions, tracking, or third-party relay.
AI 仓库情报摘要
FR-AI / ANALYSIS为什么值得关注
它通过 SSD 支持的专家流式传输和自定义 Metal 内核，在内存受限的硬件上实现了大型模型推理，并提供了完整的原生 Mac 应用、命令行工具和 OpenAI 兼容服务器，全部从零构建，不依赖 MLX 或 llama.cpp。
适合谁使用
- 拥有 8 GB 或有限 RAM 且希望在本地运行 26B 参数 LLM 的 Mac 用户
- 对高效 MoE 推理技术感兴趣的 AI 研究人员和工程师
- 希望获取设备端 AI 参考实现的 Swift 和 Metal 开发者
- 探索 Apple Silicon 本地 LLM 部署的爱好者与实验者
典型使用场景
- 在仅 8 GB 内存的 MacBook Air 上运行文本指令对话和补全
- 搭建本地回环 OpenAI 兼容服务器，用于支持工具调用的聊天应用
- 实验分块预填充和 SSD 支持的专家缓存技术以优化 MoE 模型
- 在不同 Apple Silicon Mac 上对 Gemma 4 26B-A4B 进行性能基准测试
项目优势
- 通过专家流式传输和 4 位量化，仅需约 2 GB 权重内存即可运行 26B 参数模型
- 为 Apple Silicon 优化的自定义 Metal 内核，涵盖量化 GEMV、注意力、MoE 等操作
- 详尽的文档，包括系统设计、基准结果和 102 个审核实验记录
- 流式安装器避免下载完整检查点，减少磁盘和内存占用
使用前须知
- 仅支持 macOS 26+ 和 Metal 4 的 Apple Silicon Mac，不支持 Intel 或旧版 macOS
- 仅文本推理，不支持图像、音频和视频
- 仅适用于 Gemma 4 26B-A4B 特定模型，并非像 llama.cpp 或 MLX 那样的通用框架
README 快速开始
TurboFieldfare
Gemma 4 26B-A4B inference in about 2 GB of RAM A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones.
Quick start · Local server · Benchmarks · Contribute results · How it works · Experiments · References
Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget.
TurboFieldfare runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory. It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD. This is what lets the model run on Macs with 8 GB of RAM.
The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp. The curated experiment record summarizes 102 measured results across kernels, caching, I/O, prefill, and decode.
Try it
git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac
On the first run, Swift Package Manager downloads and builds the Swift packages required by the tokenizer. The complete release build includes the foreground Mac app and its sibling decode-service executable.
When the app opens, choose Install and let TurboFieldfare fetch and repack the pinned model (about 15 GB). Once it is ready, choose Load Model, type your prompt, and press Generate.
At a glance
| Metric | Value |
|---|---|
| Model | Gemma 4 26B-A4B IT, 26B total parameters, about 3.88B active per token |
| Weights | MLX affine 4-bit, group 64; 8-bit router; 4-bit shared and routed experts |
| Memory | ~2 GB of weights and 4K KV cache |
| Storage | About 14.3 GB for the installed text-only mod |
项目描述
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
相关仓库与替代方案
根据分类、Topic 和编程语言匹配的相似项目。
didriksg
Crisp
Crisp is a free, open-source macOS menu bar app that adds HiDPI scaling, brightness control, and other display management features for external monitors, serving as an alternative to paid tools like BetterDisplay and Lunar.
makecindy
精选
cindy
Cindy is an open-source AI agent that runs locally on your machine, integrates multiple AI harnesses and models, and provides memory, skills, and automation to perform real work in your projects and apps.