DocuMind: a private AI assistant that answers from your PDFs, with page citations

The business problem

Every company runs on documents: employee handbooks, policies, contracts, product manuals, SOPs. The answers people need are in there, but finding them is slow:

• Employees ask HR or managers the same questions again and again, like "How many leave days do I have?" or "What's the hotel limit for travel?"
• Searching a 60-page PDF for one rule means skimming, guessing keywords and often missing the clause that matters.
• General-purpose AI chatbots can answer quickly, but they make things up, don't say where an answer came from, and mean uploading internal documents to a third-party cloud, which many companies can't allow.

The goal: an assistant that answers questions only from your own documents, shows the exact page it used, and runs entirely on your own machine.

Why existing solutions failed

• Ctrl+F and document search match words, not meaning. "Can I carry over leave?" won't find a clause that says "unused days can be carried forward".
• Public AI chat tools answer from general knowledge as easily as from your file, rarely cite pages, and raise data-privacy concerns.
• Enterprise knowledge platforms are expensive, need long rollouts and usually still run in someone else's cloud.

The solution

DocuMind is a private, retrieval-augmented (RAG) assistant for PDFs. Upload a document, ask questions in plain language, and get an answer grounded in the document, with the source passages and page numbers underneath.

Everything runs locally: the embedding model, the vector database and the language model (Llama 3.2 3B via Ollama). No document or question ever leaves the machine.

From PDF to searchable knowledge

1. Safe upload: only real PDFs are accepted (extension and %PDF header checked), saved under a random name and cleaned up if processing fails.
2. Page-aware extraction: text is read page by page with PyMuPDF, so every passage remembers its page number.
3. Smart chunking: pages are split into ~1,000-character chunks with a 150-character overlap, without ever cutting a word in half, so no sentence loses its context at a boundary.
4. Semantic embeddings: each chunk becomes a normalised vector using all-MiniLM-L6-v2, which matches by meaning rather than keywords.
5. Vector store: chunks go into ChromaDB (cosine similarity), tagged by document, so searches never mix up two files.

Answering a question

1. Follow-ups just work. Questions like "And what if I'm still in probation?" are rewritten into a standalone question using the last few messages, so retrieval knows what "it" or "that" refers to.
2. Retrieve: the five closest chunks from the current document are fetched.
3. Filter: if nothing is close enough, DocuMind says it couldn't find the answer instead of guessing.
4. Generate: the model answers at temperature 0 using only the retrieved passages, with each passage labelled with its page.
5. Cite: the answer is shown with the page-numbered passages it was based on.

Built to be trustworthy

• Grounded by design: the system prompt tells the model to use only the supplied context, to say when it isn't enough, and to never invent facts or page numbers.
• Prompt-injection guard: document text is wrapped and marked as untrusted data, and the model is told never to follow instructions found inside a document.
• History isn't evidence: past messages help interpret a question but can't be cited as facts about the document.
• Validated input: question length, history length and roles are checked with Pydantic before anything reaches the model.

Key features

• Chat with any PDF in plain language
• Answers with page citations and the source passages shown
• Conversation memory with question rewriting for follow-ups
• "Not in the document" responses instead of made-up answers
• Page-aware, word-safe chunking with overlap
• Semantic search with MiniLM embeddings and ChromaDB
• Runs 100% locally: Llama 3.2 3B through Ollama, with no cloud AI calls
• Prompt-injection protection and validated requests
• Clean chat UI: upload, page and chunk counts, chat with sources

Technical architecture

• Backend: Python and FastAPI, with separate services for PDF extraction, chunking, embeddings, the vector store, retrieval and the LLM. Pydantic validates every request.
• AI stack: sentence-transformers (all-MiniLM-L6-v2), ChromaDB with persistent storage, and Ollama running Llama 3.2 3B at temperature 0.
• Frontend: Next.js with React and Tailwind CSS: an upload panel, a chat with source cards and clear error states.
• Quality checks: unit tests for chunking (boundaries, overlap, invalid settings) and vector search, plus a retrieval check that asks known questions and confirms the expected page is returned.

Challenges

• Follow-up questions broke retrieval. Searching for "what about probation?" alone finds nothing useful. Rewriting the question with the conversation first, at temperature 0 and only to resolve references, fixed follow-ups without changing what the user asked.
• Chunk boundaries cut answers in half. Fixed-size splits separated rules from their conditions. Overlapping chunks that always break on a space keep each rule intact.
• Hallucinated answers. A small local model will happily guess. Strict instructions, page-labelled context and a "not found" path make it say when the document doesn't contain the answer.
• Untrusted documents. A PDF could contain text like "ignore previous instructions". The context is fenced off and explicitly treated as data, not instructions.
• Sharper source lists (next step). The model's answers are accurate, but the "Sources" panel currently shows every retrieved passage. Tightening the relevance threshold, and showing only the passages the answer actually used, is the next improvement.

The outcome

• Instant, verifiable answers. In the demo with a sample employee handbook, DocuMind answered leave, probation and policy questions correctly with the right page, and declined to invent an answer that wasn't in the document.
• Private by default. Documents, questions and answers never leave the machine, which makes it usable for HR, legal and internal knowledge that can't go to public AI tools.
• A solid base to grow: multiple documents per workspace, streaming answers, OCR for scanned PDFs, user accounts and an admin view of the questions people ask most.

What this means for other businesses

If your team keeps asking the same questions, and the answers are buried in PDFs, an assistant like DocuMind can help:

• HR and operations: policies, handbooks and SOPs answered in seconds.
• Legal and compliance: find the exact clause and page, not a paraphrase.
• Support and sales: product manuals and spec sheets turned into an always-available expert.
• Privacy-sensitive teams: everything runs on your own hardware, with no data shared with AI providers.

If your knowledge lives in documents, I can help you turn it into answers your team can trust.