drpwchen GitHub avatar

textbook-to-note

drpwchen

A local-first pipeline that converts PDF/EPUB textbooks into clean markdown, then uses AI to write structured, fully-cited notes with figures, while spending almost no LLM tokens on mechanical work.

Stars

85

7-day growth

No data

Forks

22

Open issues

0

License

MIT

Last updated

2026-07-24

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It treats each failure mode of PDF textbook processing (scrambled columns, silent data loss, figures vanishing, hallucinated citations) as a separate engineering problem with deterministic gates, and reserves the costly model only for synthesis—making high-quality, traceable notes from hundreds of books feasible on a single machine.

Who it is for

  • Medical students and clinicians managing dozens of reference textbooks
  • Researchers building a personal, searchable knowledge base from PDFs
  • Power users of AI coding agents (e.g., Claude Code) who want automated note-taking
  • Self-learners studying from multiple textbooks who want organized notes with citations

Use cases

  • Convert 40+ designated reference books into a single AI-searchable markdown corpus
  • Generate structured, per‑claim cited notes on a topic by combining multiple textbook sources
  • Extract and quality‑check figures from PDF textbooks, including tables and diagrams
  • Run cross‑book semantic search locally (via vault‑search) to find concepts without remembering which book they were in

Strengths

  • Column‑aware reading order reconstruction for two‑column textbook layouts with an exact‑fallback path
  • Deterministic QC gates for tables (page‑frame rejection, content‑loss detection, book‑level reliability banner) and figures (whitespace, text‑bleed, OCR long‑line checks) before any AI judgement
  • Hardware‑tiered OCR ladder (Surya → PaddleOCR‑VL → local vision → frontier vision) with per‑book silent‑failure detection
  • Every claim in generated notes traces to book + chapter; AI‑inferred additions are explicitly flagged

Considerations

  • Requires Python 3.10+ and manual setup of dependencies (pip, ollama, optional GPU drivers); not a one‑click app
  • Figure extraction is still marked experimental and does not handle every book layout yet; improvement PRs are welcome
  • Table extraction can produce misbindings (values in wrong rows) on continuation pages or dosage tables; the review queue (opt‑in) flags those but doesn't automatically fix them

README quick start

textbook-to-note

Turn your own PDF textbooks into an AI-searchable knowledge base and structured, fully-cited notes — figures included. A local-first pipeline that spends (almost) zero LLM tokens on the heavy lifting and reserves the frontier model for the one thing it's uniquely good at: synthesizing a note you can actually learn from.

繁體中文說明 → README.zh-TW.md

Why I built this

I've loved taking notes since med school, and over the years I've built up thousands of them — but I can no longer keep every one at the same quality by hand. In an age of information overload, trustworthy high-quality sources become the precious thing, and textbooks are among the best; yet my specialty alone has 40+ designated reference books, with a single concept scattered across chapters in many of them. Reading them all cover to cover just isn't realistic. LLMs are great at long context, but you can't dump hundreds of books on one at once — so the real unlock is pairing good search and a database with teaching the AI my own note-making process, so it produces grounded, structured notes I only have to absorb.

Left: one of the templates in templates/. Right: a real note in my vault, written against it — every claim traceable, every section where I expect it.

Which path are you on?

Not everyone wants the whole pipeline, and the parts stack in one direction — each profile below is the one above it plus one more thing. Pick the smallest one that solves your problem; you can move up later without redoing anything.

ProfileWhat you getWhat you set up
A · Converter onlyYour PDFs/EPUBs as clean, greppable markdown with page markers, tables, and figure-reference markers. Search with grep.pip install -r requirements.txt, then converter/convert.py. Nothing else — no GPU, no ollama, no index.
B · A + the note workflowAn AI writes structured, per-claim-cited notes from that corpus, with figures extracted through a QC gate.A, plus the two skills in skills/ and workflows/note-writing.md.
C · B + semantic searchCross-book retrieval by meaning rather than keyword — worth it once "which book was that in?" stops being obvious.B, plus an indexer: the companion repo vault-search, a local embed

Description

Turn your own PDF textbooks into an AI-searchable knowledge base and structured, fully-cited notes — figures included. Local-first, token-frugal.

Related repositories

Similar projects matched by category, topics, and programming language.

makecindy
Featured
makecindy GitHub avatar

cindy

Cindy is an open-source AI agent that runs locally on your machine, integrates multiple AI harnesses and models, and provides memory, skills, and automation to perform real work in your projects and apps.

AI & Machine LearningLarge Language Models
958
0xwilliamortiz
Featured
0xwilliamortiz GitHub avatar

openclaude-improved

OpenClaude is an open-source CLI coding agent that runs on any platform and supports a wide range of LLM providers, offering the same tools and workflows as Claude Code.

AI & Machine LearningLarge Language Models
577
0xwilliamortiz
Featured
0xwilliamortiz GitHub avatar

ponytail-improved

Ponytail is a plugin for AI coding agents that enforces a disciplined ladder of reuse before writing code, reducing code volume by roughly 54% while preserving safety.

AI & Machine LearningAI Agents
545