LLM System Design — Project & Lab Companion

A production-grade, interview-focused implementation of Large Language Model systems featuring:

1. project/ (llm_platform): A minimal, production-shaped LLM serving + RAG reference platform built in pure Python with zero heavy framework dependencies.
2. lab/ (llm_lab): An interactive hands-on Streamlit web application built with LangChain, backing local Ollama models (llama3.1 + nomic-embed-text) with a deterministic offline fallback.

---

📁 Repository Layout

LLM-system-design/
├── project/                  # Reference LLM serving + RAG platform (pure Python)
│   ├── llm_platform/         # Core platform modules (pipeline, router, RAG, agents, etc.)
│   ├── tests/                # Offline unit test suite (150+ tests)
│   ├── run_demo.py           # End-to-end demo script with telemetry
│   └── README.md
└── lab/                      # Interactive web lab (LangChain + Streamlit)
    ├── llm_lab/              # Lab modules & LangChain implementations
    │   └── platform_bridge.py  # Bridges all reference modules to the UI
    ├── tests/                # Lab test suite
    ├── app.py                # Streamlit web dashboard
    └── README.md

---

🚀 Quick Start

1. Running the Reference Platform (project/)

(Dependency-free, pure Python standard library; optional numpy/FastAPI)

cd project
python run_demo.py          # Run end-to-end platform demo with telemetry
python -m pytest -q         # Run full offline test suite (150+ tests)

Optional: Launch HTTP API Server

pip install -r requirements.txt
uvicorn llm_platform.app:app --reload
# POST http://127.0.0.1:8000/v1/chat   {"query": "What is the KV cache?"}

Optional: Connect to Real OpenAI-Compatible Endpoints (vLLM, TGI, Ollama, Vendor)

export OPENAI_BASE_URL="https://api.your-provider.com/v1"
export OPENAI_API_KEY="sk-..."
python run_demo.py

---

2. Running the Interactive UI Lab (lab/)

(LangChain + Streamlit UI with local Ollama models or offline fallback)

cd lab
pip install -r requirements.txt

# (Optional) Real models via Ollama:
ollama pull llama3.1
ollama pull nomic-embed-text
ollama serve            # if not already running

# Generate seed datasets (also runnable from UI)
python -m llm_lab.seed

# Launch Streamlit Web UI
streamlit run app