Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2605.20246 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910244671586304 |
|---|---|
| author | Wu, Xiongbin Luo, Zhihao Lei, Shanzhe Zhang, Lechao Wang, Xuhong Yang, Jie Zheng, Zhonglong Zheng, Yuanjie Tan, Xin Liu, Wei |
| author_facet | Wu, Xiongbin Luo, Zhihao Lei, Shanzhe Zhang, Lechao Wang, Xuhong Yang, Jie Zheng, Zhonglong Zheng, Yuanjie Tan, Xin Liu, Wei |
| contents | Recently, vision-language model (VLM) agents have shown promising progress in open-world tasks, where successful task completion often requires multiple turns of visual perception and action execution. However, existing methods still rely primarily on Supervised Fine-Tuning (SFT) with expert demonstrations, while the advanced reinforcement learning (RL) algorithm, specifically Group Relative Policy Optimization (GRPO), has not been effectively employed for multi-turn RL in these tasks because standard GRPO requires full trajectories as training samples which leads to excessively long context and noise. To address this issue, we propose GROW, a RL framework for open-world VLM agents that decomposes collected trajectories into state-action samples, and computes advantages between these samples rather than treating a full trajectory as a single entity. We further provide a surrogate analysis indicating that, even though the grouped samples are conditioned on different local states rather than an identical prompt context, the objective can preserve the core relative policy optimization signal of GRPO under simplifying assumptions. Experiments on more than 800 Minecraft tasks show that our method achieves state-of-the-art (SOTA) performance, demonstrating the effectiveness of our proposed RL framework for open-world VLM agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_20246 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents Wu, Xiongbin Luo, Zhihao Lei, Shanzhe Zhang, Lechao Wang, Xuhong Yang, Jie Zheng, Zhonglong Zheng, Yuanjie Tan, Xin Liu, Wei Machine Learning Artificial Intelligence Recently, vision-language model (VLM) agents have shown promising progress in open-world tasks, where successful task completion often requires multiple turns of visual perception and action execution. However, existing methods still rely primarily on Supervised Fine-Tuning (SFT) with expert demonstrations, while the advanced reinforcement learning (RL) algorithm, specifically Group Relative Policy Optimization (GRPO), has not been effectively employed for multi-turn RL in these tasks because standard GRPO requires full trajectories as training samples which leads to excessively long context and noise. To address this issue, we propose GROW, a RL framework for open-world VLM agents that decomposes collected trajectories into state-action samples, and computes advantages between these samples rather than treating a full trajectory as a single entity. We further provide a surrogate analysis indicating that, even though the grouped samples are conditioned on different local states rather than an identical prompt context, the objective can preserve the core relative policy optimization signal of GRPO under simplifying assumptions. Experiments on more than 800 Minecraft tasks show that our method achieves state-of-the-art (SOTA) performance, demonstrating the effectiveness of our proposed RL framework for open-world VLM agents. |
| title | GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.20246 |