jamesob
精选
local-llm
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.
AI 仓库情报摘要
FR-AI / ANALYSIS为什么值得关注
它提供了完全本地化、免费的云端问答替代方案，强调隐私性和可定制性，所有组件均为开源。
适合谁使用
- 构建文档问答工具的开发者
- 研究 RAG 管线的研究人员
- 需要本地文档查询且不想付费 API 的用户
- 探索本地 AI 助手的爱好者
典型使用场景
- 对公司内部文档进行问答
- 离线查询学术论文
- 搜索个人本地知识库
- 对敏感数据进行离线文档分析
项目优势
- 无需付费 API，完全本地运行
- 支持多种文档格式（PDF、TXT、MD）
- 通过 FAISS 实现快速语义检索
- 嵌入模型和 LLM 模型易于替换
使用前须知
- 仅有命令行界面，尚无图形或网页界面
- 不支持扫描版 PDF 的 OCR
- 当前版本缺少对话记忆和混合搜索功能
README 快速开始
📚 RAG Document Q&A System
An AI-powered Retrieval-Augmented Generation (RAG) application that answers questions using your own documents instead of relying only on the language model's knowledge. The system processes PDF, TXT, and Markdown files, retrieves the most relevant document sections using semantic search, and generates accurate responses with Qwen2.5.
🚀 Features
- 📄 Supports PDF, TXT, and Markdown documents
- 🔍 Semantic document retrieval using FAISS
- 🧠 Sentence Transformer embeddings
- 🤖 Open-source Qwen2.5 language model
- 💬 Interactive command-line chat mode
- ⚡ Fast local vector database
- 📊 Retrieval evaluation support
- 🔄 Easily replaceable embedding and LLM models
🏗️ Architecture
Documents (PDF / TXT / MD)
│
▼
Document Chunking
│
▼
Sentence Transformer Embeddings
│
▼
FAISS Vector Database
│
▼
User Question
│
▼
Semantic Similarity Search
│
▼
Top Relevant Document Chunks
│
▼
Qwen2.5 Instruction Model
│
▼
Context-Aware AI Response
📂 Project Structure
rag-document-qa
│
├── src
│ ├── ingest.py
│ ├── rag_pipeline.py
│ └── evaluate.py
│
├── requirements.txt
├── README.md
└── documents/
🛠️ Technologies Used
- Python
- LangChain
- Hugging Face Transformers
- Sentence Transformers
- FAISS
- Qwen2.5
- PyTorch
📦 Installation
Clone the repository
git clone https://github.com/yourusername/rag-document-qa.git
Move to the project folder
cd rag-document-qa
Create a virtual environment
python -m venv venv
Activate it
Windows
venv\Scripts\activate
Linux / macOS
source venv/bin/activate
Install dependencies
pip install -r requirements.txt
🚀 Usage
1. Ingest Documents
python src/ingest.py
2. Start the RAG Chatbot
python src/rag_pipeline.p
项目描述
AI-powered Retrieval-Augmented Generation (RAG) Document Q&A system using LangChain, FAISS, Sentence Transformers, and Qwen2.5 to answer questions from PDF, TXT, and Markdown documents with accurate, context-aware responses.
相关仓库与替代方案
根据分类、Topic 和编程语言匹配的相似项目。
slvDev
精选
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
gavamedia
精选
deltafin
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.