Saturday, 21 November 2026 Press Releases Newsletter RSS
Breaking
WhatsApp Business Account Hacked in Pakistan: Full Recovery Guide Special Technology Zones Authority (STZA) & IT Park Expansion in Pakistan: Islamabad, Karachi, and Lahore Ground Report Pakistan Software Export Board & Industry Consortium Launch 'TechBridge Pakistan 2026' to Connect 500 Startups With Global Markets Global Tech Safety Summit 2026: International Standards for Frontier AI and Data Security Announced WhatsApp Business Account Hacked in Pakistan: Full Recovery Guide Special Technology Zones Authority (STZA) & IT Park Expansion in Pakistan: Islamabad, Karachi, and Lahore Ground Report Pakistan Software Export Board & Industry Consortium Launch 'TechBridge Pakistan 2026' to Connect 500 Startups With Global Markets Global Tech Safety Summit 2026: International Standards for Frontier AI and Data Security Announced
WritePures

A working pipeline that retrieves the wrong passages produces confident wrong answers. This chapter is about the retrieval quality that determines everything downstream.

How many chunks to return

Three to five is the usual sweet spot.

Too few and you miss the passage containing the answer. Too many and you fill the context window with noise, which both slows generation and measurably degrades answers — models are worse at finding a fact buried in the middle of a long context than at the edges.

Start at 5 and tune on real questions.

Where pure vector search fails

Embeddings capture meaning, which is exactly why they are bad at exact strings.

Search for invoice number INV-2026-4471 and vector search returns passages that are semantically about invoices — not the one with that number. Same for product codes, surnames, error codes and legal citations.

The fix is hybrid search: run both a keyword search and a vector search, then merge.

pip install rank-bm25
from rank_bm25 import BM25Okapi

corpus = [r["text"] for r in records]
bm25 = BM25Okapi([doc.lower().split() for doc in corpus])

def hybrid_search(question: str, k: int = 5):
    # Keyword side
    keyword_scores = bm25.get_scores(question.lower().split())
    keyword_top = sorted(range(len(corpus)), key=lambda i: -keyword_scores[i])[:k]

    # Vector side
    vector_hits = collection.query(query_embeddings=[embed_query(question)], n_results=k)
    vector_ids = vector_hits["ids"][0]

    # Reciprocal rank fusion: rank position matters, raw scores do not compare
    scores = {}
    for rank, idx in enumerate(keyword_top):
        scores[records[idx]["id"]] = scores.get(records[idx]["id"], 0) + 1 / (60 + rank)
    for rank, doc_id in enumerate(vector_ids):
        scores[doc_id] = scores.get(doc_id, 0) + 1 / (60 + rank)

    return sorted(scores, key=scores.get, reverse=True)[:k]

Reciprocal rank fusion is the standard way to merge two rankings whose scores are not on the same scale. The constant 60 is conventional and not magic.

Filtering with metadata

The metadata you stored in chapter two earns its keep here:

collection.query(
    query_embeddings=[embed_query("notice period")],
    n_results=5,
    where={"source": {"$contains": "contracts/"}},
)

Narrowing to a folder, a date range or a document type before searching is often a bigger quality win than any model change.

Reranking, if you need more

If retrieval is still imprecise, retrieve more candidates than you need — say 20 — then score each one against the question with a small model and keep the best 5. It costs a second and noticeably sharpens results on hard queries.

Test retrieval on its own

This is the discipline that makes debugging possible. Before involving the language model at all, write ten real questions and check whether the right passage appears in the top five.

If it does not, no prompt engineering will save you — the model never sees the answer. Fixing retrieval first, in isolation, is the difference between a system you can improve and one you can only guess at.