RAG & LLMs / 4. VECTOR STORE

Stage 4: Vector Store

Persisting embeddings and enabling fast ANN search

EXPLANATION

A vector store persists your embeddings and enables fast approximate nearest-neighbor (ANN) search. When a query comes in, it's embedded and compared against all stored vectors.

Options by use case:
• ChromaDB → local, zero config, perfect for dev & moderate scale
• FAISS → in-memory, blazing fast, Facebook's library
• Pinecone → fully managed cloud, production scale
• pgvector → PostgreSQL extension, if you're already on Postgres

Start with ChromaDB. Migrate to Pinecone only when you actually need to.

DATA FLOW

chunks + their embeddings
          ↓
   ChromaDB (persisted to ./chroma_db)
  ┌───────────────────────────────────────────────────┐
  │  id │ vector (1024d)    │ text         │ metadata │
  │  0  │ [0.23,-0.87,...]  │ "BERT is..." │ {p:1}    │
  │  1  │ [0.11, 0.44,...]  │ "RAG uses..."│ {p:2}    │
  │  2  │ [-0.9, 0.02,...]  │ "The attn..."│ {p:3}    │
  └───────────────────────────────────────────────────┘
          ↑
  query vector → cosine similarity → top-k results

CODE

PYTHON

1from langchain_community.vectorstores import Chroma, FAISS
2from pathlib import Path
3
4CHROMA_PATH = "./chroma_db"
5
6# ── Build ChromaDB from chunks (run once) ─────────────────────────
7vectorstore = Chroma.from_documents(
8    documents=chunks,
9    embedding=embeddings,
10    persist_directory=CHROMA_PATH,
11    collection_name="deepdocs_rag",
12)
13print(f"Indexed: {vectorstore._collection.count()} chunks")
14
15# ── Load existing store (skip re-indexing on restarts) ────────────
16if Path(CHROMA_PATH).exists():
17    vectorstore = Chroma(
18        persist_directory=CHROMA_PATH,
19        embedding_function=embeddings,
20        collection_name="deepdocs_rag",
21    )
22
23# ── Basic similarity search ───────────────────────────────────────
24query = "How does attention mechanism work?"
25results = vectorstore.similarity_search(query, k=5)
26for i, doc in enumerate(results):
27    print(f"[{i+1}] {doc.page_content[:120]}...")
28    print(f"     Source: {doc.metadata['source']}")
29
30# ── Search with similarity scores ────────────────────────────────
31results_with_scores = vectorstore.similarity_search_with_score(query, k=5)
32for doc, score in results_with_scores:
33    print(f"Score: {score:.4f} | {doc.page_content[:80]}...")
34
35# ── MMR: diverse results (avoids redundant chunks) ────────────────
36diverse = vectorstore.max_marginal_relevance_search(
37    query, k=5, fetch_k=20, lambda_mult=0.5,
38)
39
40# ── FAISS alternative ─────────────────────────────────────────────
41# faiss_store = FAISS.from_documents(chunks, embeddings)
42# faiss_store.save_local("./faiss_index")
43# faiss_store = FAISS.load_local("./faiss_index", embeddings)

← PREV3. Embedding (Bi-Encoder)NEXT →5. Retrieval