RAG
Embedding
An embedding is a vector of numbers representing the meaning of a piece of text, so that similar meanings sit near each other in that space.
Retrieval works by embedding the query and finding the stored vectors nearest to it. The vector is an average of everything in the chunk, which is why chunk quality determines retrieval quality.
Embedding is almost always the cheap part of a RAG system. The generation call that consumes the retrieved chunks typically costs an order of magnitude more.
In practice
Embeddings are strong on meaning and weak exactly where users are precise — product codes, error numbers, version strings, surnames. "ERR_5521" and "ERR_5522" sit close together in vector space and are completely different problems. That is why keyword search alongside vector search is not redundancy, it is coverage of the queries embeddings handle worst.
Common questions
Can I mix embeddings from different models?
No. Vectors from different models occupy different spaces, so similarity between them is meaningless. Changing embedding model means re-embedding the entire corpus, which is the cost people forget when planning a switch.
Why does semantic search miss exact matches?
Embeddings encode meaning, and "ERR_5521" and "ERR_5522" mean almost the same thing while being completely different problems. Keyword search alongside vector search covers exactly the queries embeddings handle worst.