BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910151466811392 |
|---|---|
| author | Liu, Shiyu Yin, Yongjing Yan, Jianhao Tang, Yunbo Zhang, Qinggang Li, Bei Chen, Xin Wang, Jingang Cai, Xunliang Su, Jinsong |
| author_facet | Liu, Shiyu Yin, Yongjing Yan, Jianhao Tang, Yunbo Zhang, Qinggang Li, Bei Chen, Xin Wang, Jingang Cai, Xunliang Su, Jinsong |
| contents | RL-based agentic search enables LLMs to solve complex questions via dynamic planning and external search. While this approach significantly enhances accuracy with agent policies optimized via large-scale reinforcement learning, we identify a critical gap in reliability: these agents fail to recognize their reasoning boundaries and rarely admit ``I DON'T KNOW'' (IDK) even when evidence is insufficient or reasoning reaches its limit. The lack of reliability often leads to plausible but unreliable answers, introducing significant risks in many real-world scenarios. To this end, we propose Boundary-Aware Policy Optimization (BAPO), a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy. BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut. Extensive experiments on four benchmarks demonstrate that BAPO substantially enhances the overall reliability of agentic search. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_11037 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search Liu, Shiyu Yin, Yongjing Yan, Jianhao Tang, Yunbo Zhang, Qinggang Li, Bei Chen, Xin Wang, Jingang Cai, Xunliang Su, Jinsong Artificial Intelligence RL-based agentic search enables LLMs to solve complex questions via dynamic planning and external search. While this approach significantly enhances accuracy with agent policies optimized via large-scale reinforcement learning, we identify a critical gap in reliability: these agents fail to recognize their reasoning boundaries and rarely admit ``I DON'T KNOW'' (IDK) even when evidence is insufficient or reasoning reaches its limit. The lack of reliability often leads to plausible but unreliable answers, introducing significant risks in many real-world scenarios. To this end, we propose Boundary-Aware Policy Optimization (BAPO), a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy. BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut. Extensive experiments on four benchmarks demonstrate that BAPO substantially enhances the overall reliability of agentic search. |
| title | BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2601.11037 |