Another impressive paper by Google DeepMind.
It takes a closer look at the limits of embedding-based retrieval.
If you work with vector embeddings, bookmark this one.
Let's break down the technical details:
Quick Overview
This paper looks at how search engines that rely on vector embeddings have built-in limits.
Even if you train them perfectly, they just can’t handle every possible search query once the combinations of relevant documents get too complex.
The authors prove this with math, then confirm it with experiments on a simple but tricky dataset they call LIMIT.
This paper looks at how search engines that rely on vector embeddings have built-in limits.
Even if you train them perfectly, they just can’t handle every possible search query once the combinations of relevant documents get too complex.
The authors prove this with math, then confirm it with experiments on a simple but tricky dataset they call LIMIT.
Built-in ceiling
Each document and query is turned into a single vector.
The study shows there’s only so many correct top-k results these vectors can represent.
If you ask for more combinations than the vectors can encode, it’s impossible for the system to get it right.
Each document and query is turned into a single vector.
The study shows there’s only so many correct top-k results these vectors can represent.
If you ask for more combinations than the vectors can encode, it’s impossible for the system to get it right.
Best-case test
Even when the vectors are directly optimized on the test answers (an unrealistic cheating setup), the number of documents they can handle grows roughly as a cubic function of the embedding size (d).
For real-world scale (millions of documents), even very large embeddings fall short.
Even when the vectors are directly optimized on the test answers (an unrealistic cheating setup), the number of documents they can handle grows roughly as a cubic function of the embedding size (d).
For real-world scale (millions of documents), even very large embeddings fall short.
LIMIT dataset
The authors built a toy dataset where queries are super simple. LIMIT maps all 2-document combinations to natural-language queries like “Who likes X?”
Despite this, top embedding models collapse, scoring below 20% recall when the task forces them to juggle all possible two-document combinations.
A smaller 46-document version is still unsolved at recall@20.
The authors built a toy dataset where queries are super simple. LIMIT maps all 2-document combinations to natural-language queries like “Who likes X?”
Despite this, top embedding models collapse, scoring below 20% recall when the task forces them to juggle all possible two-document combinations.
A smaller 46-document version is still unsolved at recall@20.
Combination density matters
When the dataset is made maximally dense (every possible pairing must be handled), performance nosedives.
Sparser setups (like random, cycle, or disjoint patterns) are easier, which shows the problem is about the number of combinations, not language difficulty.
When the dataset is made maximally dense (every possible pairing must be handled), performance nosedives.
Sparser setups (like random, cycle, or disjoint patterns) are easier, which shows the problem is about the number of combinations, not language difficulty.
Alternatives
Cross-encoders (which compare queries to every document directly), multi-vector retrievers, and sparse models like BM25 don’t suffer from the same ceiling or limitations.
These could be better choices when queries involve lots of concept mixing.
Cross-encoders (which compare queries to every document directly), multi-vector retrievers, and sparse models like BM25 don’t suffer from the same ceiling or limitations.
These could be better choices when queries involve lots of concept mixing.
Concluding remarks
In conclusion, single-vector embeddings are powerful, but fundamentally limited.
If your searches require mixing and matching lots of concepts (as instruction-following queries often do), you’ll eventually hit a wall no matter how much data or training you throw at them.
Paper: arxiv.org/abs/2508.21038
In conclusion, single-vector embeddings are powerful, but fundamentally limited.
If your searches require mixing and matching lots of concepts (as instruction-following queries often do), you’ll eventually hit a wall no matter how much data or training you throw at them.
Paper: arxiv.org/abs/2508.21038
Generated by Thread Navigator
Press ⌘ + S to quick-export