ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909800970846208 |
|---|---|
| author | Chen, Mingyang Sun, Linzhuang Li, Tianpeng Sun, Haoze Zhou, Yijie Zhu, Chenzheng Wang, Haofen Pan, Jeff Z. Zhang, Wen Chen, Huajun Yang, Fan Zhou, Zenan Chen, Weipeng |
| author_facet | Chen, Mingyang Sun, Linzhuang Li, Tianpeng Sun, Haoze Zhou, Yijie Zhu, Chenzheng Wang, Haofen Pan, Jeff Z. Zhang, Wen Chen, Huajun Yang, Fan Zhou, Zenan Chen, Weipeng |
| contents | Large Language Models (LLMs) have shown remarkable capabilities in reasoning, exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating reasoning with external search processes remains challenging, especially for complex multi-hop questions requiring multiple retrieval steps. We propose ReSearch, a novel framework that trains LLMs to Reason with Search via reinforcement learning without using any supervised data on reasoning steps. Our approach treats search operations as integral components of the reasoning chain, where when and how to perform searches is guided by text-based thinking, and search results subsequently influence further reasoning. We train ReSearch on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct extensive experiments. Despite being trained on only one dataset, our models demonstrate strong generalizability across various benchmarks. Analysis reveals that ReSearch naturally elicits advanced reasoning capabilities such as reflection and self-correction during the reinforcement learning process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_19470 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning Chen, Mingyang Sun, Linzhuang Li, Tianpeng Sun, Haoze Zhou, Yijie Zhu, Chenzheng Wang, Haofen Pan, Jeff Z. Zhang, Wen Chen, Huajun Yang, Fan Zhou, Zenan Chen, Weipeng Artificial Intelligence Computation and Language Large Language Models (LLMs) have shown remarkable capabilities in reasoning, exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating reasoning with external search processes remains challenging, especially for complex multi-hop questions requiring multiple retrieval steps. We propose ReSearch, a novel framework that trains LLMs to Reason with Search via reinforcement learning without using any supervised data on reasoning steps. Our approach treats search operations as integral components of the reasoning chain, where when and how to perform searches is guided by text-based thinking, and search results subsequently influence further reasoning. We train ReSearch on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct extensive experiments. Despite being trained on only one dataset, our models demonstrate strong generalizability across various benchmarks. Analysis reveals that ReSearch naturally elicits advanced reasoning capabilities such as reflection and self-correction during the reinforcement learning process. |
| title | ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2503.19470 |