Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915469899857920 |
|---|---|
| author | Deng, Yong Wang, Guoqing Ying, Zhenzhe Wu, Xiaofeng Lin, Jinzhen Xiong, Wenwen Dai, Yuqin Yang, Shuo Zhang, Zhanwei Wang, Qiwen Qin, Yang Wang, Yuan Zha, Quanxing Dai, Sunhao Meng, Changhua |
| author_facet | Deng, Yong Wang, Guoqing Ying, Zhenzhe Wu, Xiaofeng Lin, Jinzhen Xiong, Wenwen Dai, Yuqin Yang, Shuo Zhang, Zhanwei Wang, Qiwen Qin, Yang Wang, Yuan Zha, Quanxing Dai, Sunhao Meng, Changhua |
| contents | Large language models (LLMs) exhibit remarkable problem-solving abilities, but struggle with complex tasks due to static internal knowledge. Retrieval-Augmented Generation (RAG) enhances access to external information, yet remains limited in multi-hop reasoning and strategic search due to rigid workflows. Recent advancements in agentic deep research empower LLMs to autonomously reason, search, and synthesize information. However, current approaches relying on outcome-based reinforcement learning (RL) face critical issues such as conflicting gradients and reward sparsity, limiting performance gains and training efficiency. To address these, we first propose Atomic Thought, a novel LLM thinking paradigm that decomposes reasoning into fine-grained functional units. These units are supervised by Reasoning Reward Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained guidance. Building on this, we propose Atom-Searcher, a novel RL framework for agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher uses a curriculum-inspired reward schedule, prioritizing process-level ATR early and transitioning to outcome rewards, accelerating convergence on effective reasoning paths. Experiments on seven benchmarks show consistent improvements over the state-of-the-art. Key advantages include: (1) Atom-Searcher scales computation at test-time. (2) Atomic Thought provides supervision anchors for RRMs, bridging deep research tasks and RRMs. (3) Atom-Searcher exhibits more interpretable, human-like reasoning patterns. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_12800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward Deng, Yong Wang, Guoqing Ying, Zhenzhe Wu, Xiaofeng Lin, Jinzhen Xiong, Wenwen Dai, Yuqin Yang, Shuo Zhang, Zhanwei Wang, Qiwen Qin, Yang Wang, Yuan Zha, Quanxing Dai, Sunhao Meng, Changhua Computation and Language Artificial Intelligence Large language models (LLMs) exhibit remarkable problem-solving abilities, but struggle with complex tasks due to static internal knowledge. Retrieval-Augmented Generation (RAG) enhances access to external information, yet remains limited in multi-hop reasoning and strategic search due to rigid workflows. Recent advancements in agentic deep research empower LLMs to autonomously reason, search, and synthesize information. However, current approaches relying on outcome-based reinforcement learning (RL) face critical issues such as conflicting gradients and reward sparsity, limiting performance gains and training efficiency. To address these, we first propose Atomic Thought, a novel LLM thinking paradigm that decomposes reasoning into fine-grained functional units. These units are supervised by Reasoning Reward Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained guidance. Building on this, we propose Atom-Searcher, a novel RL framework for agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher uses a curriculum-inspired reward schedule, prioritizing process-level ATR early and transitioning to outcome rewards, accelerating convergence on effective reasoning paths. Experiments on seven benchmarks show consistent improvements over the state-of-the-art. Key advantages include: (1) Atom-Searcher scales computation at test-time. (2) Atomic Thought provides supervision anchors for RRMs, bridging deep research tasks and RRMs. (3) Atom-Searcher exhibits more interpretable, human-like reasoning patterns. |
| title | Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2508.12800 |