LangChain RAG Pipeline

EXPLANATION

RAG (Retrieval Augmented Generation) grounds your LLM in real data. Instead of hallucinating, the model retrieves relevant context first, then generates answers only from that context.

The pipeline has two phases:
• Indexing Phase (offline): Load → Split → Embed → Store
• Query Phase (online): Retrieve → Rerank → Generate

LangChain provides clean abstractions for every stage — you can swap any component without rewriting the pipeline.

DATA FLOW

┌──────────────────────────────────────────────────────────┐
  │               INDEXING PHASE (offline)                   │
  │                                                          │
  │  PDFs → DocumentLoader → TextSplitter → EmbeddingModel   │
  │                                              ↓           │
  │                                        VectorStore       │
  └──────────────────────────────────────────────────────────┘
                               ↕
  ┌──────────────────────────────────────────────────────────┐
  │                QUERY PHASE (online)                      │
  │                                                          │
  │  Query → BiEncoder → VectorStore → Top-10 chunks         │
  │                           ↓                              │
  │                  CrossEncoder.rerank() → Top-3           │
  │                           ↓                              │
  │              LLM.generate(context + query) → Answer      │
  └──────────────────────────────────────────────────────────┘

CODE

1# Install all dependencies
2pip install langchain langchain-community langchain-huggingface
3pip install chromadb sentence-transformers rank-bm25
4pip install pypdf google-generativeai langchain-google-genai
5
6# Sanity check
7python -c "import langchain; print(langchain.__version__)"