GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918149872418816 |
|---|---|
| author | Chen, Cong Ji, Kaixiang Zhong, Hao Zhu, Muzhi Li, Anzhou Gan, Guo Huang, Ziyuan Zou, Cheng Liu, Jiajia Chen, Jingdong Chen, Hao Shen, Chunhua |
| author_facet | Chen, Cong Ji, Kaixiang Zhong, Hao Zhu, Muzhi Li, Anzhou Gan, Guo Huang, Ziyuan Zou, Cheng Liu, Jiajia Chen, Jingdong Chen, Hao Shen, Chunhua |
| contents | Autonomous agents for long-sequence Graphical User Interface tasks are hindered by sparse rewards and the intractable credit assignment problem. To address these challenges, we introduce GUI-Shepherd, a Process Reward Model that provides dense, step-by-step feedback to guide agents. GUI-Shepherd is trained on a diverse large-scale data set of $52$k interactions that features human-annotated scores and GPT-4o generated rationales, enabling it to serve both as a reward provider for RL training and as a verifier for inference. As far as we know, we are the first to conduct a systematic study of process supervision in GUI agents, across diverse settings from online long-horizon tasks to offline single-step prediction. On the online AndroidWorld benchmark, GUI-Shepherd improves success rate by $7.7$ points via multi-turn online PPO, significantly outperforming Outcome Reward Model based competitors. When used as an inference verifier, it brings $5.1$ points improvements. The benefits generalize to the offline AndroidControl benchmark, with gains of $2.2$ points as a reward provider and $4.3$ points as a verifier. Collectively, our results establish that high-fidelity process supervision is critical for building more capable GUI agents and present a generalizable solution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23738 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks Chen, Cong Ji, Kaixiang Zhong, Hao Zhu, Muzhi Li, Anzhou Gan, Guo Huang, Ziyuan Zou, Cheng Liu, Jiajia Chen, Jingdong Chen, Hao Shen, Chunhua Artificial Intelligence Autonomous agents for long-sequence Graphical User Interface tasks are hindered by sparse rewards and the intractable credit assignment problem. To address these challenges, we introduce GUI-Shepherd, a Process Reward Model that provides dense, step-by-step feedback to guide agents. GUI-Shepherd is trained on a diverse large-scale data set of $52$k interactions that features human-annotated scores and GPT-4o generated rationales, enabling it to serve both as a reward provider for RL training and as a verifier for inference. As far as we know, we are the first to conduct a systematic study of process supervision in GUI agents, across diverse settings from online long-horizon tasks to offline single-step prediction. On the online AndroidWorld benchmark, GUI-Shepherd improves success rate by $7.7$ points via multi-turn online PPO, significantly outperforming Outcome Reward Model based competitors. When used as an inference verifier, it brings $5.1$ points improvements. The benefits generalize to the offline AndroidControl benchmark, with gains of $2.2$ points as a reward provider and $4.3$ points as a verifier. Collectively, our results establish that high-fidelity process supervision is critical for building more capable GUI agents and present a generalizable solution. |
| title | GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Tasks |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.23738 |