Before writing any code, it is worth understanding the pipeline, because almost every RAG problem you will hit is a failure in one specific stage and knowing which one saves hours.
The four steps
1. Chunk. Split your documents into passages of a few hundred words. You cannot search a whole 80-page PDF usefully; you need pieces small enough that one is actually about something.
2. Embed. Convert each chunk into a list of numbers — a vector — that represents its meaning. Passages about similar things end up with similar vectors, even when they share no words.
3. Retrieve. When a question arrives, embed the question the same way, then find the chunks whose vectors are closest to it. Those are your candidate answers.
4. Generate. Paste those chunks into a prompt along with the question, and tell the model to answer using only the material provided.
That is the whole technique. No training, no GPUs beyond what you already have, and the index updates the moment you add a file.
Why this beats keyword search
A keyword search for "termination notice period" misses a contract clause that says "either party may end this agreement on thirty days' written notice". There is no word overlap at all. Embeddings capture meaning, so the clause is found anyway.
This is also why RAG works across phrasings, synonyms and sometimes languages — the vector represents the idea, not the spelling.
Why not fine-tune instead
This is the most common wrong turn, so it is worth being direct.
Fine-tuning adjusts a model's weights on your data. It is good at teaching style and behaviour — respond in this format, adopt this tone, follow this workflow. It is poor at teaching facts you need to retrieve reliably.
| RAG | Fine-tuning | |
|---|---|---|
| Adding a document | Instant | Retrain |
| Cost | Near zero | Significant |
| Can cite sources | Yes | No |
| Correcting an error | Edit the file | Retrain |
| Good for | Facts and documents | Style and behaviour |
For "answer questions about my documents", RAG wins on every row that matters. The two are also not exclusive — serious systems sometimes do both — but start here.
What RAG does not fix
It does not make a small model smarter. If a 7B model cannot reason through a complex question, giving it better source material helps but does not change its reasoning ability.
It also does not guarantee the model uses what you gave it. Models sometimes answer from training data anyway, or blend the two. Chapter seven deals with forcing the model to stay on the provided material and say so when the answer is not there.