You now have vectors. You need somewhere to put them that can answer "which of these is closest to this query" quickly.
What to use
| Store | Good for | Cost of entry |
|---|---|---|
| Chroma | Local work, single machine | pip install, no Docker |
| Qdrant | Production, larger collections | A Docker container |
| pgvector | You already run Postgres | An extension |
| A NumPy array | Under ~5,000 chunks | Nothing |
Start with Chroma. It persists to disk, needs no server, and is easy to back up — the directory is the database. Moving to Qdrant later is a contained change because the interface is similar.
And be honest about scale: under a few thousand chunks, comparing every vector with NumPy takes milliseconds. You do not need a vector database to search your own notes.
Loading into Chroma
pip install chromadb
import chromadb
client = chromadb.PersistentClient(path="./vector-store")
collection = client.get_or_create_collection(
name="documents",
metadata={"hnsw:space": "cosine"},
)
collection.add(
ids=[r["id"] for r in records],
embeddings=[r["vector"] for r in records],
documents=[r["text"] for r in records],
metadatas=[{"source": r["source"], "chunk_index": r["chunk_index"]} for r in records],
)
print(collection.count())
Two details worth noticing. PersistentClient writes to disk, so
your index survives a restart — the in-memory client silently loses everything
and is a common first-time surprise. And setting cosine distance explicitly
matters, because it is the right measure for text embeddings and not always
the default.
Searching it
results = collection.query(
query_embeddings=[embed_query("What is the notice period?")],
n_results=5,
)
for text, meta, distance in zip(
results["documents"][0], results["metadatas"][0], results["distances"][0]
):
print(f"[{distance:.3f}] {meta['source']}")
print(text[:200], "\n")
Note embed_query, not embed_document — the prefix
from the last chapter applies here.
Keeping it current
Re-embedding everything whenever one file changes is wasteful. Store a hash of each chunk's text and skip anything unchanged:
import hashlib
def content_hash(text: str) -> str:
return hashlib.sha256(text.encode()).hexdigest()[:16]
Keep the hash in the metadata, compare on import, and only embed what is new. On a folder that changes daily this turns an hour-long reindex into seconds.
Backing it up
With Chroma the whole store is a directory. Copy it, and you have a backup. That is worth doing before any reindex, because a chunking change you dislike is otherwise an hour of embedding to undo.