## Definition **Query rewriting** (also called query reformulation or query expansion) is the pre-retrieval step of transforming the user's raw query into a form that retrieves more relevant documents. It is especially important in multi-turn conversations, where the user's latest message is often elliptical and only meaningful in context. ## The Core Problem In a conversational RAG system: ``` User: When was the last time John Doe bought something from us? AI: John last bought a Fruity Fedora hat two weeks ago. User: How about Emily Doe? ``` The final query "How about Emily Doe?" is meaningless to a retriever with no conversation context. Sending it verbatim returns irrelevant documents. The query must be rewritten to "When was the last time Emily Doe bought something from us?" before retrieval. ## Approaches ### Heuristic rewriting In traditional search engines, query rewriting is done with handcrafted rules: spell correction, synonym expansion, entity disambiguation. Fast and predictable, but brittle for open-ended conversation. ### LLM-based rewriting Use a language model — typically a cheaper, faster model than the main generator — to rewrite the query: ``` System: Given the conversation below, rewrite the last user message so it makes sense as a standalone retrieval query. Conversation: [history] Last message: [user input] Rewritten query: ``` The rewriting model must handle cases where the query is unanswerable without additional lookup (e.g., "How about his wife?" requires first resolving who "his wife" is). If the identity cannot be resolved, the rewriter should signal that rather than hallucinate a name. ## Why a Separate Rewriting Step The retrieval stage and the generation stage have different needs: - **Retrieval** needs a dense, self-contained query that matches document language. - **Generation** needs the full conversational context for a coherent response. Separating them lets each component be optimised independently. A small, fast rewriting model adds minimal latency while substantially improving retrieval precision. ## Relation to Other Retrieval Tactics Query rewriting sits upstream of the retrieval pipeline and is compatible with all retrieval algorithms: - With [[Term-Based Retrieval]], a better-formed query improves keyword matching. - With [[Embedding-Based Retrieval]], a self-contained query produces a more accurate embedding. - With [[Hybrid Search]], both benefit simultaneously. Related techniques in the embedding-based space include HyDE (Hypothetical Document Embeddings) and multi-query retrieval, which generate multiple alternative phrasings to broaden the retrieval net. ## Related - [[Retrieval-Augmented Generation]] - [[Embedding-Based Retrieval]] - [[Term-Based Retrieval]] - [[Hybrid Search]] - [[In-Context Learning]] ## Sources - [[AI Engineering - Chip Huyen]]