A local model knows what it was trained on and nothing else. It has never seen your contracts, your meeting notes, your product manual or your research folder. This tutorial fixes that — without uploading a single file anywhere.
The technique is called retrieval-augmented generation, and the idea is simpler than the name. When you ask a question, you first search your own documents for the few passages that look relevant, then paste those into the prompt and ask the model to answer from them. The model supplies the language; your documents supply the facts.
Done properly it also tells you which document an answer came from, which turns an unverifiable chatbot response into something you can check.
What you need before starting
- Ollama installed with a working model — our local models tutorial covers this from scratch.
- Python 3.10 or later, and comfort installing packages with pip.
- A folder of documents you actually want to search. Start with 20–50 files rather than ten thousand.
What you will have at the end
A script you can point at a folder, which indexes it once and then answers questions about it with citations — entirely offline, at no cost per query, with your documents never leaving the machine.