Skill Reuse as Compression in Agentic RL
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911733504802816 |
|---|---|
| author | Xu, Zhikun Feng, Yu Dineen, Jacob Shi, Taiwei Zhao, Jieyu Zhou, Ben |
| author_facet | Xu, Zhikun Feng, Yu Dineen, Jacob Shi, Taiwei Zhao, Jieyu Zhou, Ben |
| contents | Large language model agents trained with reinforcement learning (RL) often learn brittle, task-specific shortcuts. We hypothesize that agents generalize better when their successful trajectories are structurally compressible, decomposed into a small set of reusable abstract patterns. To formalize this, we introduce ReuseRL, which grounds agentic RL in the Minimum Description Length (MDL) principle. ReuseRL extracts a shared skill dictionary from successful trajectories and augments the RL objective with a segmentation cost, explicitly penalizing idiosyncratic behaviors that encode poorly. We prove a PAC-Bayes generalization bound for this compression penalty. Across ALFWorld, TextWorld-Cooking, and Countdown-Stepwise, ReuseRL improves in- and out-of-distribution success over vanilla GRPO and strong round-length baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_31509 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Skill Reuse as Compression in Agentic RL Xu, Zhikun Feng, Yu Dineen, Jacob Shi, Taiwei Zhao, Jieyu Zhou, Ben Machine Learning Artificial Intelligence Large language model agents trained with reinforcement learning (RL) often learn brittle, task-specific shortcuts. We hypothesize that agents generalize better when their successful trajectories are structurally compressible, decomposed into a small set of reusable abstract patterns. To formalize this, we introduce ReuseRL, which grounds agentic RL in the Minimum Description Length (MDL) principle. ReuseRL extracts a shared skill dictionary from successful trajectories and augments the RL objective with a segmentation cost, explicitly penalizing idiosyncratic behaviors that encode poorly. We prove a PAC-Bayes generalization bound for this compression penalty. Across ALFWorld, TextWorld-Cooking, and Countdown-Stepwise, ReuseRL improves in- and out-of-distribution success over vanilla GRPO and strong round-length baselines. |
| title | Skill Reuse as Compression in Agentic RL |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.31509 |