An enterprise-grade FastAPI backend for a conversational AI chatbot with session memory, Retrieval-Augmented Generation over PDFs using LangChain and ChromaDB, plus source citations.

Stars

35

7-day growth

No data

Forks

0

Open issues

0

License

No data

Last updated

2026-07-27

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It provides a ready-to-use RAG chatbot backend with memory and vector search, structured for extensibility (auth, streaming, tools), and includes a clear setup pipeline from PDF ingestion to API endpoints.

Who it is for

  • Developers building AI chatbots with document question-answering
  • Teams seeking a FastAPI + LangChain starter for RAG systems
  • Engineers experimenting with vector databases and LLM memory
  • Prototypers needing a minimal yet extensible backend for conversational AI

Use cases

  • Customer support chatbot that answers from company manuals (PDFs)
  • Internal knowledge base assistant with session-based conversation history
  • Educational tool that retrieves and cites specific pages from textbooks
  • Rapid prototype for a document-grounded QA system with streaming support

Strengths

  • Clear, step-by-step setup with virtual environment, dependency install, PDF ingestion, and server launch
  • Modular folder structure (auth, memory, rag, tools, analytics, streaming) enabling easy extension
  • Built-in source citations alongside answers for transparency and trust
  • Utilizes proven components: FastAPI for API, LangChain for orchestration, ChromaDB for vector search

Considerations

  • Requires an OpenAI API key, limiting to OpenAI models and incurring usage costs
  • ChromaDB is single-node and may need replacement for large-scale production deployments
  • No frontend provided; only a backend API with Swagger docs
  • Security features (auth folder) are not described – likely minimal or placeholder

README quick start

🧠 AI RAG Chatbot Backend

Memory • Retrieval-Augmented Generation (RAG) • FastAPI • LangChain • Vector DB


📌 Project Overview

This project implements an enterprise-grade backend for a Conversational AI Chatbot using FastAPI and LangChain.

The chatbot supports:

  • ✅ Session-based conversational memory
  • ✅ Retrieval Augmented Generation (RAG) using PDFs
  • ✅ Vector search using ChromaDB
  • ✅ Source citations with answers
  • ✅ Extensible architecture (Auth, Streaming, Tools ready)

🏗️ Architecture

Client → FastAPI → LangChain (Memory + RAG) → ChromaDB → Documents


🧰 Tech Stack

  • FastAPI
  • LangChain (classic + openai)
  • OpenAI
  • ChromaDB
  • PyPDF
  • Uvicorn
  • Python 3.10+

📁 Folder Structure

ai_rag_chatbot_backend/
├── app.py
├── config.py
├── database.py
├── models.py
├── auth/
├── memory/
├── rag/
├── tools/
├── analytics/
├── streaming/
├── utils/
├── data/
├── vectorstore/
├── .env
└── README.md

🔐 Environment Variables

Create a .env file:

OPENAI_API_KEY=your_openai_api_key_here
CHROMA_PERSIST_DIR=vectorstore

▶️ Step-by-Step Setup

1️⃣ Create Virtual Environment

python -m venv venv
venv\Scripts\activate

2️⃣ Install Dependencies

pip install fastapi uvicorn pydantic python-dotenv langchain langchain-classic langchain-openai langchain-community chromadb pypdf

3️⃣ Add PDFs

Place PDFs inside:

data/

4️⃣ Ingest Documents

python rag/ingest.py

5️⃣ Run Server

uvicorn app:app --reload

Open:

http://127.0.0.1:8000/docs

🧪 Test API

POST /chat

{
  "message": "What is RAG?",
  "session_id": "test-1",
  "memory_type": "buffer"
}

Related repositories

Similar projects matched by category, topics, and programming language.

slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
gavamedia
Featured
gavamedia GitHub avatar

deltafin

Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.

AI & Machine LearningLarge Language Models
304
jamesob
Featured
jamesob GitHub avatar

local-llm

A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.

AI & Machine LearningLarge Language Models
1,660