UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917361787863040 |
|---|---|
| author | Lin, Zichuan Liu, Feiyu Yang, Yijun Lyu, Jiafei Gao, Yiming Liu, Yicheng Lu, Zhicong Yu, Yangbin Yang, Mingyu Li, Junyou Ye, Deheng Jiang, Jie |
| author_facet | Lin, Zichuan Liu, Feiyu Yang, Yijun Lyu, Jiafei Gao, Yiming Liu, Yicheng Lu, Zhicong Yu, Yangbin Yang, Mingyu Li, Junyou Ye, Deheng Jiang, Jie |
| contents | Autonomous mobile GUI agents have attracted increasing attention along with the advancement of Multimodal Large Language Models (MLLMs). However, existing methods still suffer from inefficient learning from failed trajectories and ambiguous credit assignment under sparse rewards for long-horizon GUI tasks. To that end, we propose UI-Voyager, a novel two-stage self-evolving mobile GUI agent. In the first stage, we employ Rejection Fine-Tuning (RFT), which enables the continuous co-evolution of data and models in a fully autonomous loop. The second stage introduces Group Relative Self-Distillation (GRSD), which identifies critical fork points in group rollouts and constructs dense step-level supervision from successful trajectories to correct failed ones. Extensive experiments on AndroidWorld show that our 4B model achieves an 81.0% Pass@1 success rate, outperforming numerous recent baselines and exceeding human-level performance. Ablation and case studies further verify the effectiveness of GRSD. Our method represents a significant leap toward efficient, self-evolving, and high-performance mobile GUI automation without expensive manual data annotation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_24533 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience Lin, Zichuan Liu, Feiyu Yang, Yijun Lyu, Jiafei Gao, Yiming Liu, Yicheng Lu, Zhicong Yu, Yangbin Yang, Mingyu Li, Junyou Ye, Deheng Jiang, Jie Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Autonomous mobile GUI agents have attracted increasing attention along with the advancement of Multimodal Large Language Models (MLLMs). However, existing methods still suffer from inefficient learning from failed trajectories and ambiguous credit assignment under sparse rewards for long-horizon GUI tasks. To that end, we propose UI-Voyager, a novel two-stage self-evolving mobile GUI agent. In the first stage, we employ Rejection Fine-Tuning (RFT), which enables the continuous co-evolution of data and models in a fully autonomous loop. The second stage introduces Group Relative Self-Distillation (GRSD), which identifies critical fork points in group rollouts and constructs dense step-level supervision from successful trajectories to correct failed ones. Extensive experiments on AndroidWorld show that our 4B model achieves an 81.0% Pass@1 success rate, outperforming numerous recent baselines and exceeding human-level performance. Ablation and case studies further verify the effectiveness of GRSD. Our method represents a significant leap toward efficient, self-evolving, and high-performance mobile GUI automation without expensive manual data annotation. |
| title | UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.24533 |