SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911433744187392 |
|---|---|
| author | Xia, Peng Chen, Jianwen Wang, Hanyang Liu, Jiaqi Zeng, Kaide Wang, Yu Han, Siwei Zhou, Yiyang Zhao, Xujiang Chen, Haifeng Zheng, Zeyu Xie, Cihang Yao, Huaxiu |
| author_facet | Xia, Peng Chen, Jianwen Wang, Hanyang Liu, Jiaqi Zeng, Kaide Wang, Yu Han, Siwei Zhou, Yiyang Zhao, Xujiang Chen, Haifeng Zheng, Zeyu Xie, Cihang Yao, Huaxiu |
| contents | Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_08234 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning Xia, Peng Chen, Jianwen Wang, Hanyang Liu, Jiaqi Zeng, Kaide Wang, Yu Han, Siwei Zhou, Yiyang Zhao, Xujiang Chen, Haifeng Zheng, Zeyu Xie, Cihang Yao, Huaxiu Machine Learning Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL. |
| title | SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.08234 |