What an Embedding Is
An embedding model converts text into a vector — typically several hundred to a few thousand numbers. Texts with similar meaning produce vectors that sit close together in that space.
The useful property is that closeness captures meaning rather than wording. "How do I cancel my subscription" and "I want to stop my membership" share almost no words and produce nearby vectors.
This is what makes semantic search possible, and it is why embeddings underpin RAG, recommendation, clustering and classification.
Similarity Search
Finding relevant text becomes a geometry problem: embed the query, then find the stored vectors nearest to it, usually by cosine similarity.
At small scale you can compare against every stored vector. At millions of vectors that is too slow, so vector databases use approximate nearest neighbour algorithms — HNSW is the most common — which trade a small amount of accuracy for very large speed gains.
What Vector Databases Do
A vector database stores embeddings alongside metadata and supports fast similarity search with filtering.
The filtering matters more than people expect. Real queries are rarely pure similarity — they are "find similar content, but only from this customer, in this language, published after this date". Combining vector search with metadata filters efficiently is the actual engineering problem.
Options include dedicated databases such as Pinecone, Weaviate, Qdrant and Milvus, and vector extensions to existing databases such as pgvector for PostgreSQL. For most applications, adding vectors to a database you already run is simpler than introducing a new one.
Choosing and Using an Embedding Model
Embedding models differ in dimensionality, cost, speed and domain suitability. Larger is not automatically better — higher dimensionality increases storage and search cost for sometimes marginal accuracy gain.
The critical rule: index and query must use the same embedding model. Changing the model means re-embedding the entire corpus. Choose deliberately, because migration is expensive.
General-purpose models work well for general text and noticeably less well for specialised domains with distinctive vocabulary — legal, medical, scientific. Domain-specific models exist and are worth evaluating for those cases.
Where Embeddings Are Used Beyond RAG
Semantic search on a website, returning results that match meaning rather than keywords.
Recommendation — finding items similar to what a user engaged with.
Deduplication and clustering — grouping near-identical support tickets, reviews or documents.
Classification — assigning categories by similarity to labelled examples, often more robustly than keyword rules.
Chunking, Which Decides More Than the Model Does
Retrieval quality is determined more by how documents are split than by which embedding model produced the vectors, and chunking is where most effort should go.
Too small and a chunk loses the context that made it meaningful — a sentence referring to something defined two paragraphs earlier retrieves without its subject. Too large and the relevant passage is diluted by surrounding text, so the match is weaker and the model receives more to ignore.
What works better than a fixed size: split on structure — headings, sections, list items — so chunks correspond to units of meaning rather than counts of characters. Overlap adjacent chunks so a passage spanning a boundary appears whole somewhere. Keep the heading with the chunk, which is the cheapest single improvement most systems are missing. And store the source and position so the answer can be cited back to a location a person can check.
Test chunking changes against a fixed question set, because it is easy to improve one query and quietly worsen ten.
Why Pure Similarity Search Disappoints
Semantic search finds text that means something similar, which is not always text that answers the question — and the gap shows up in predictable places.
Exact terms. A product code, an error number, a person's name. Embeddings are poor at exactness, and a keyword index is better at it. Hybrid search — combining semantic and keyword scoring — is the standard answer and fixes most complaints about retrieval quality.
Negation and comparison. Which of these does not support X is close in embedding space to text saying it does.
Recency and filtering. Similarity has no opinion about dates or permissions, so filter on metadata before or alongside the search rather than hoping the ranking handles it.
Reranking — passing the top results through a more expensive model that scores them directly against the query — usually improves quality more than changing the embedding model, and is the second thing to try after hybrid search.
The practical sequence when retrieval disappoints: fix chunking, add keyword search, add reranking, and only then consider a different embedding model. Most teams do that list backwards.