R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913741944127488 |
|---|---|
| author | Song, Huatong Jiang, Jinhao Min, Yingqian Chen, Jie Chen, Zhipeng Zhao, Wayne Xin Fang, Lei Wen, Ji-Rong |
| author_facet | Song, Huatong Jiang, Jinhao Min, Yingqian Chen, Jie Chen, Zhipeng Zhao, Wayne Xin Fang, Lei Wen, Ji-Rong |
| contents | Existing Large Reasoning Models (LRMs) have shown the potential of reinforcement learning (RL) to enhance the complex reasoning capabilities of Large Language Models~(LLMs). While they achieve remarkable performance on challenging tasks such as mathematics and coding, they often rely on their internal knowledge to solve problems, which can be inadequate for time-sensitive or knowledge-intensive questions, leading to inaccuracies and hallucinations. To address this, we propose \textbf{R1-Searcher}, a novel two-stage outcome-based RL approach designed to enhance the search capabilities of LLMs. This method allows LLMs to autonomously invoke external search systems to access additional knowledge during the reasoning process. Our framework relies exclusively on RL, without requiring process rewards or distillation for a cold start. % effectively generalizing to out-of-domain datasets and supporting both Base and Instruct models. Our experiments demonstrate that our method significantly outperforms previous strong RAG methods, even when compared to the closed-source GPT-4o-mini. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05592 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning Song, Huatong Jiang, Jinhao Min, Yingqian Chen, Jie Chen, Zhipeng Zhao, Wayne Xin Fang, Lei Wen, Ji-Rong Artificial Intelligence Computation and Language Information Retrieval Existing Large Reasoning Models (LRMs) have shown the potential of reinforcement learning (RL) to enhance the complex reasoning capabilities of Large Language Models~(LLMs). While they achieve remarkable performance on challenging tasks such as mathematics and coding, they often rely on their internal knowledge to solve problems, which can be inadequate for time-sensitive or knowledge-intensive questions, leading to inaccuracies and hallucinations. To address this, we propose \textbf{R1-Searcher}, a novel two-stage outcome-based RL approach designed to enhance the search capabilities of LLMs. This method allows LLMs to autonomously invoke external search systems to access additional knowledge during the reasoning process. Our framework relies exclusively on RL, without requiring process rewards or distillation for a cold start. % effectively generalizing to out-of-domain datasets and supporting both Base and Instruct models. Our experiments demonstrate that our method significantly outperforms previous strong RAG methods, even when compared to the closed-source GPT-4o-mini. |
| title | R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning |
| topic | Artificial Intelligence Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2503.05592 |