Embedding Models
What are Embeddings?
Embeddings are dense vector representations of text where semantic similarity is captured by geometric proximity. Similar meaning → similar vectors → small cosine distance. The embedding model is the most critical component of RAG after chunking — bad embeddings means bad retrieval regardless of your vector DB.
Popular Embedding Models
all-MiniLM-L6-v2 — 384 dimensions, 22M params. Fast, free, good quality. Best for: local/offline RAG. all-mpnet-base-v2 — 768 dimensions, 110M params. Better quality than MiniLM, slower. text-embedding-3-small (OpenAI) — 1536 dims. Excellent quality, paid API. bge-large-en-v1.5 (BAAI) — 1024 dims. State-of-the-art for English, free. bge-m3 (BAAI) — Multilingual, multi-granularity. Best open-source embedding model. Voyage-3 (Anthropic) — Best-in-class for RAG with Claude, designed to complement Anthropic models.
Embedding Evaluation (MTEB)
MTEB (Massive Text Embedding Benchmark) is the standard leaderboard for embedding models. Covers retrieval, classification, clustering, semantic similarity across 56 tasks. Check the latest leaderboard at huggingface.co/spaces/mteb/leaderboard before choosing an embedding model.
Embedding Models in Practice
Query embeddings and document embeddings are often computed differently (asymmetric retrieval). Always check the model's recommended usage.
Finished reading? Mark it complete to earn your XP.