StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915989764964352 |
|---|---|
| author | Xue, Xiangyuan Zhou, Yifan Wang, Zidong Tang, Shengji Torr, Philip Ouyang, Wanli Bai, Lei Yin, Zhenfei |
| author_facet | Xue, Xiangyuan Zhou, Yifan Wang, Zidong Tang, Shengji Torr, Philip Ouyang, Wanli Bai, Lei Yin, Zhenfei |
| contents | Large language models (LLMs) are increasingly used as interactive agents, but optimizing them for long-horizon decision making remains difficult because current methods are largely purely reactive, which weakens both exploration and credit assignment over extended trajectories. In this work, we present Strategic Trajectory Abstraction (StraTA), a simple framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning (RL). StraTA samples a compact strategy from the initial task state, conditions subsequent actions on that strategy, and trains strategy generation and action execution jointly with a hierarchical GRPO-style rollout design, further enhanced by diverse strategy rollout and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance over strong baselines. StraTA reaches success rates of 93.1% on ALFWorld and 84.2% on WebShop. On SciWorld, StraTA attains a 63.5% overall score, outperforming frontier closed-source models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_06642 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction Xue, Xiangyuan Zhou, Yifan Wang, Zidong Tang, Shengji Torr, Philip Ouyang, Wanli Bai, Lei Yin, Zhenfei Computation and Language Artificial Intelligence Large language models (LLMs) are increasingly used as interactive agents, but optimizing them for long-horizon decision making remains difficult because current methods are largely purely reactive, which weakens both exploration and credit assignment over extended trajectories. In this work, we present Strategic Trajectory Abstraction (StraTA), a simple framework that introduces an explicit trajectory-level strategy into agentic reinforcement learning (RL). StraTA samples a compact strategy from the initial task state, conditions subsequent actions on that strategy, and trains strategy generation and action execution jointly with a hierarchical GRPO-style rollout design, further enhanced by diverse strategy rollout and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld show that StraTA consistently improves both sample efficiency and final performance over strong baselines. StraTA reaches success rates of 93.1% on ALFWorld and 84.2% on WebShop. On SciWorld, StraTA attains a 63.5% overall score, outperforming frontier closed-source models. |
| title | StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2605.06642 |