SE-Search: Self-Evolving Search Agent via Memory and Dense Reward
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917311588335616 |
|---|---|
| author | Li, Jian Jin, Yizhang Liu, Dongqi Ding, Hang Wu, Jiafu Chen, Dongsheng Shen, Yunhang Qin, Yulei Tai, Ying Wang, Chengjie Yuan, Xiaotong Wang, Yabiao |
| author_facet | Li, Jian Jin, Yizhang Liu, Dongqi Ding, Hang Wu, Jiafu Chen, Dongsheng Shen, Yunhang Qin, Yulei Tai, Ying Wang, Chengjie Yuan, Xiaotong Wang, Yabiao |
| contents | Retrieval augmented generation (RAG) reduces hallucinations and factual errors in large language models (LLMs) by conditioning generation on retrieved external knowledge. Recent search agents further cast RAG as an autonomous, multi-turn information-seeking process. However, existing methods often accumulate irrelevant or noisy documents and rely on sparse reinforcement learning signals. We propose \textbf{S}elf-\textbf{E}volving \textbf{Search}, a Self-Evolving Search agent that improves online search behavior through three components, memory purification, atomic query training, and dense rewards. SE-Search follows a \textit{Think-Search-Memorize} strategy that retains salient evidence while filtering irrelevant content. Atomic query training promotes shorter and more diverse queries, improving evidence acquisition. Dense rewards provide fine-grained feedback that speeds training. Experiments on single-hop and multi-hop question answering benchmarks show that \texttt{SE-Search-3B} outperforms strong baselines, yielding a $10.8$ point absolute improvement and a $33.8\%$ relative gain over Search-R1.\footnote{We will make the code and model weights publicly available upon acceptance.} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_03293 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SE-Search: Self-Evolving Search Agent via Memory and Dense Reward Li, Jian Jin, Yizhang Liu, Dongqi Ding, Hang Wu, Jiafu Chen, Dongsheng Shen, Yunhang Qin, Yulei Tai, Ying Wang, Chengjie Yuan, Xiaotong Wang, Yabiao Computation and Language Retrieval augmented generation (RAG) reduces hallucinations and factual errors in large language models (LLMs) by conditioning generation on retrieved external knowledge. Recent search agents further cast RAG as an autonomous, multi-turn information-seeking process. However, existing methods often accumulate irrelevant or noisy documents and rely on sparse reinforcement learning signals. We propose \textbf{S}elf-\textbf{E}volving \textbf{Search}, a Self-Evolving Search agent that improves online search behavior through three components, memory purification, atomic query training, and dense rewards. SE-Search follows a \textit{Think-Search-Memorize} strategy that retains salient evidence while filtering irrelevant content. Atomic query training promotes shorter and more diverse queries, improving evidence acquisition. Dense rewards provide fine-grained feedback that speeds training. Experiments on single-hop and multi-hop question answering benchmarks show that \texttt{SE-Search-3B} outperforms strong baselines, yielding a $10.8$ point absolute improvement and a $33.8\%$ relative gain over Search-R1.\footnote{We will make the code and model weights publicly available upon acceptance.} |
| title | SE-Search: Self-Evolving Search Agent via Memory and Dense Reward |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.03293 |