esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
它提供了一个即用型 RAG 聊天机器人后端,具备记忆和向量搜索功能,结构易于扩展(认证、流式、工具),并包含从 PDF 摄取到 API 端点的清晰设置流程。
Memory • Retrieval-Augmented Generation (RAG) • FastAPI • LangChain • Vector DB
This project implements an enterprise-grade backend for a Conversational AI Chatbot using FastAPI and LangChain.
The chatbot supports:
Client → FastAPI → LangChain (Memory + RAG) → ChromaDB → Documents
ai_rag_chatbot_backend/
├── app.py
├── config.py
├── database.py
├── models.py
├── auth/
├── memory/
├── rag/
├── tools/
├── analytics/
├── streaming/
├── utils/
├── data/
├── vectorstore/
├── .env
└── README.md
Create a .env file:
OPENAI_API_KEY=your_openai_api_key_here
CHROMA_PERSIST_DIR=vectorstore
python -m venv venv
venv\Scripts\activate
pip install fastapi uvicorn pydantic python-dotenv langchain langchain-classic langchain-openai langchain-community chromadb pypdf
Place PDFs inside:
data/
python rag/ingest.py
uvicorn app:app --reload
Open:
http://127.0.0.1:8000/docs
POST /chat
{
"message": "What is RAG?",
"session_id": "test-1",
"memory_type": "buffer"
}
根据分类、Topic 和编程语言匹配的相似项目。
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.

A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.