## Definition **Reranking** is a second, more precise pass over a retrieval shortlist: a fast first stage fetches many candidates, then a slower, more accurate model **re-scores** them and keeps the best few. It is the "cheap-and-wide, then careful-and-narrow" pattern at the heart of production retrieval (the precision stage in [[Hybrid Search]]). ## Bi-encoder vs cross-encoder — the key distinction - First-stage retrieval uses a **bi-encoder**: the query and each document are embedded **separately** and compared by [[Semantic Search|cosine similarity]]. Fast enough to search millions of documents, but the model never sees the pair *together*. - A **cross-encoder reranker** feeds the query and a candidate through the model **together**, producing a much better relevance score — but far too slow to run over the whole corpus. The two-stage design gets the best of both: a bi-encoder narrows millions to dozens; a cross-encoder carefully orders those dozens. ## The two-stage pattern Retrieve top-k cheaply (e.g. 50) → rerank → keep top-n for the prompt (e.g. 5). A large quality gain for a small extra cost, because the expensive model only ever sees the shortlist. ## Why inclusion matters more than exact rank When the result feeds an LLM, what matters most is that the truly relevant document is **not dropped** before the model sees it; absolute order matters less — though [[Lost in the Middle Effect|position in the context]] still counts. ## Related - [[Hybrid Search]] - [[Embedding-Based Retrieval]] - [[Retrieval-Augmented Generation]] - [[Lost in the Middle Effect]] ## Sources - [[AI Engineering - Chip Huyen]]