jamesob
Featured
local-llm
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It provides a fully local, free alternative to cloud-based Q&A services, emphasizing privacy and customizability with open-source components.
Who it is for
- Developers building document Q&A tools
- Researchers experimenting with RAG pipelines
- Users needing private document querying without API costs
- Hobbyists exploring local AI assistants
Use cases
- Question answering on corporate internal documents
- Academic paper querying without internet
- Personal knowledge base search from local files
- Offline document analysis for sensitive data
Strengths
- No paid APIs required—fully local
- Support for multiple document formats (PDF, TXT, MD)
- Fast semantic retrieval via FAISS
- Easily replaceable embedding and LLM models
Considerations
- Command-line interface only—no GUI or web interface yet
- Lacks OCR for scanned PDFs
- No conversation memory or hybrid search in current version
README quick start
📚 RAG Document Q&A System
An AI-powered Retrieval-Augmented Generation (RAG) application that answers questions using your own documents instead of relying only on the language model's knowledge. The system processes PDF, TXT, and Markdown files, retrieves the most relevant document sections using semantic search, and generates accurate responses with Qwen2.5.
🚀 Features
- 📄 Supports PDF, TXT, and Markdown documents
- 🔍 Semantic document retrieval using FAISS
- 🧠 Sentence Transformer embeddings
- 🤖 Open-source Qwen2.5 language model
- 💬 Interactive command-line chat mode
- ⚡ Fast local vector database
- 📊 Retrieval evaluation support
- 🔄 Easily replaceable embedding and LLM models
🏗️ Architecture
Documents (PDF / TXT / MD)
│
▼
Document Chunking
│
▼
Sentence Transformer Embeddings
│
▼
FAISS Vector Database
│
▼
User Question
│
▼
Semantic Similarity Search
│
▼
Top Relevant Document Chunks
│
▼
Qwen2.5 Instruction Model
│
▼
Context-Aware AI Response
📂 Project Structure
rag-document-qa
│
├── src
│ ├── ingest.py
│ ├── rag_pipeline.py
│ └── evaluate.py
│
├── requirements.txt
├── README.md
└── documents/
🛠️ Technologies Used
- Python
- LangChain
- Hugging Face Transformers
- Sentence Transformers
- FAISS
- Qwen2.5
- PyTorch
📦 Installation
Clone the repository
git clone https://github.com/yourusername/rag-document-qa.git
Move to the project folder
cd rag-document-qa
Create a virtual environment
python -m venv venv
Activate it
Windows
venv\Scripts\activate
Linux / macOS
source venv/bin/activate
Install dependencies
pip install -r requirements.txt
🚀 Usage
1. Ingest Documents
python src/ingest.py
2. Start the RAG Chatbot
python src/rag_pipeline.p
Description
AI-powered Retrieval-Augmented Generation (RAG) Document Q&A system using LangChain, FAISS, Sentence Transformers, and Qwen2.5 to answer questions from PDF, TXT, and Markdown documents with accurate, context-aware responses.
Related repositories
Similar projects matched by category, topics, and programming language.
slvDev
Featured
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
gavamedia
Featured
deltafin
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.