ReasonIR: Training Retrievers for Reasoning Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916712484438016 |
|---|---|
| author | Shao, Rulin Qiao, Rui Kishore, Varsha Muennighoff, Niklas Lin, Xi Victoria Rus, Daniela Low, Bryan Kian Hsiang Min, Sewon Yih, Wen-tau Koh, Pang Wei Zettlemoyer, Luke |
| author_facet | Shao, Rulin Qiao, Rui Kishore, Varsha Muennighoff, Niklas Lin, Xi Victoria Rus, Daniela Low, Bryan Kian Hsiang Min, Sewon Yih, Wen-tau Koh, Pang Wei Zettlemoyer, Luke |
| contents | We present ReasonIR-8B, the first retriever specifically trained for general reasoning tasks. Existing retrievers have shown limited gains on reasoning tasks, in part because existing training datasets focus on short factual queries tied to documents that straightforwardly answer them. We develop a synthetic data generation pipeline that, for each document, our pipeline creates a challenging and relevant query, along with a plausibly related but ultimately unhelpful hard negative. By training on a mixture of our synthetic data and existing public data, ReasonIR-8B achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used reasoning-intensive information retrieval (IR) benchmark. When applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4% and 22.6% respectively, relative to the closed-book baseline, outperforming other retrievers and search engines. In addition, ReasonIR-8B uses test-time compute more effectively: on BRIGHT, its performance consistently increases with longer and more information-rich rewritten queries; it continues to outperform other retrievers when combined with an LLM reranker. Our training recipe is general and can be easily extended to future LLMs; to this end, we open-source our code, data, and model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20595 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ReasonIR: Training Retrievers for Reasoning Tasks Shao, Rulin Qiao, Rui Kishore, Varsha Muennighoff, Niklas Lin, Xi Victoria Rus, Daniela Low, Bryan Kian Hsiang Min, Sewon Yih, Wen-tau Koh, Pang Wei Zettlemoyer, Luke Artificial Intelligence Computation and Language Information Retrieval Machine Learning We present ReasonIR-8B, the first retriever specifically trained for general reasoning tasks. Existing retrievers have shown limited gains on reasoning tasks, in part because existing training datasets focus on short factual queries tied to documents that straightforwardly answer them. We develop a synthetic data generation pipeline that, for each document, our pipeline creates a challenging and relevant query, along with a plausibly related but ultimately unhelpful hard negative. By training on a mixture of our synthetic data and existing public data, ReasonIR-8B achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used reasoning-intensive information retrieval (IR) benchmark. When applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4% and 22.6% respectively, relative to the closed-book baseline, outperforming other retrievers and search engines. In addition, ReasonIR-8B uses test-time compute more effectively: on BRIGHT, its performance consistently increases with longer and more information-rich rewritten queries; it continues to outperform other retrievers when combined with an LLM reranker. Our training recipe is general and can be easily extended to future LLMs; to this end, we open-source our code, data, and model. |
| title | ReasonIR: Training Retrievers for Reasoning Tasks |
| topic | Artificial Intelligence Computation and Language Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2504.20595 |