Hybrid Search — Dense + Sparse
Dense vs Sparse Retrieval
Dense (vector) retrieval — Semantic similarity via embeddings. Finds conceptually related docs even without keyword match. Example: 'car' matches 'automobile'. Sparse (keyword) retrieval — BM25/TF-IDF exact keyword matching. Great for rare terms, product names, codes, identifiers. Example: 'GPT-4o' won't get confused with 'GPT-4'. Hybrid — Combines both. Gets the best of semantic understanding AND keyword precision. Standard in production RAG.
BM25 — Best Matching 25
BM25 is the gold standard keyword retrieval algorithm, improving on TF-IDF: Score(q,d) = Σ IDF(qᵢ) · (tf(qᵢ,d) · (k₁+1)) / (tf(qᵢ,d) + k₁·(1 - b + b·|d|/avgdl)) k₁=1.5 (term frequency saturation), b=0.75 (document length normalization) BM25 is still competitive with dense retrieval on many benchmarks — especially for exact phrase matching.
Reciprocal Rank Fusion (RRF)
RRF merges ranked lists from multiple retrieval systems without needing score normalization: RRF_score(d) = Σ 1 / (k + rank(d)) k=60 is standard (smoothing constant). Simple but very effective for combining dense and sparse results.
Hybrid Search Implementation
For production, use Weaviate or Elasticsearch which support hybrid search natively with BM25 + vector fusion built-in.
Finished reading? Mark it complete to earn your XP.