Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.21534 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911495014580224 |
|---|---|
| author | Wang, Xiaoxuan Zhang, Han Wang, Haixin Shi, Yidan Li, Ruoyan Han, Kaiqiao Tong, Chenyi Deng, Haoran Sun, Renliang Taylor, Alexander Zhu, Yanqiao Cong, Jason Sun, Yizhou Wang, Wei |
| author_facet | Wang, Xiaoxuan Zhang, Han Wang, Haixin Shi, Yidan Li, Ruoyan Han, Kaiqiao Tong, Chenyi Deng, Haoran Sun, Renliang Taylor, Alexander Zhu, Yanqiao Cong, Jason Sun, Yizhou Wang, Wei |
| contents | Agentic reinforcement learning (ARL) has rapidly gained attention as a promising paradigm for training agents to solve complex, multi-step interactive tasks. Despite encouraging early results, ARL remains highly unstable, often leading to training collapse. This instability limits scalability to larger environments and longer interaction horizons, and constrains systematic exploration of algorithmic design choices. In this paper, we first propose ARLArena, a stable training recipe and systematic analysis framework that examines training stability in a controlled and reproducible setting. ARLArena first constructs a clean and standardized testbed. Then, we decompose policy gradient into four core design dimensions and assess the performance and stability of each dimension. Through this fine-grained analysis, we distill a unified perspective on ARL and propose SAMPO, a stable agentic policy optimization method designed to mitigate the dominant sources of instability in ARL. Empirically, SAMPO achieves consistently stable training and strong performance across diverse agentic tasks. Overall, this study provides a unifying policy gradient perspective for ARL and offers practical guidance for building stable and reproducible LLM-based agent training pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21534 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning Wang, Xiaoxuan Zhang, Han Wang, Haixin Shi, Yidan Li, Ruoyan Han, Kaiqiao Tong, Chenyi Deng, Haoran Sun, Renliang Taylor, Alexander Zhu, Yanqiao Cong, Jason Sun, Yizhou Wang, Wei Artificial Intelligence Agentic reinforcement learning (ARL) has rapidly gained attention as a promising paradigm for training agents to solve complex, multi-step interactive tasks. Despite encouraging early results, ARL remains highly unstable, often leading to training collapse. This instability limits scalability to larger environments and longer interaction horizons, and constrains systematic exploration of algorithmic design choices. In this paper, we first propose ARLArena, a stable training recipe and systematic analysis framework that examines training stability in a controlled and reproducible setting. ARLArena first constructs a clean and standardized testbed. Then, we decompose policy gradient into four core design dimensions and assess the performance and stability of each dimension. Through this fine-grained analysis, we distill a unified perspective on ARL and propose SAMPO, a stable agentic policy optimization method designed to mitigate the dominant sources of instability in ARL. Empirically, SAMPO achieves consistently stable training and strong performance across diverse agentic tasks. Overall, this study provides a unifying policy gradient perspective for ARL and offers practical guidance for building stable and reproducible LLM-based agent training pipelines. |
| title | ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.21534 |