RAG — Retrieval-Augmented Generation
What is RAG?
RAG (Retrieval-Augmented Generation) combines a retrieval system with an LLM generator. When a user asks a question, the system first retrieves relevant documents from a knowledge base, then injects them into the LLM prompt as context before generating a response. This solves three critical LLM problems: • Knowledge cutoff — LLMs have stale training data; RAG provides fresh documents • Hallucinations — LLMs invent facts; RAG grounds answers in real retrieved text • Private knowledge — LLMs can't know your proprietary data; RAG injects it dynamically
RAG vs Fine-tuning
Fine-tuning bakes knowledge into weights — expensive, hard to update, still hallucinates. RAG keeps knowledge external — cheap, instantly updatable, verifiable with citations. When to fine-tune instead: • Changing the model's style or tone (not knowledge) • Very specific domain vocabulary or format • When you need sub-100ms latency with no retrieval step
Two Phases of RAG
Offline Indexing (done once, updated as docs change): 1. Load documents (PDFs, web pages, databases, APIs) 2. Chunk into segments (256–1024 tokens) 3. Embed each chunk with an embedding model 4. Store vectors in a vector database Online Querying (happens at request time): 1. Embed the user query 2. Search vector DB for top-k similar chunks 3. Inject retrieved chunks into LLM prompt 4. LLM generates a grounded answer
Complete RAG Pipeline
normalize_embeddings=True + IndexFlatIP is equivalent to cosine similarity search — more stable than L2 distance for text embeddings.
Finished reading? Mark it complete to earn your XP.