Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915956708605952 |
|---|---|
| author | Gu, Jihao Ai, Qihang Wang, Yingyao Bu, Pi Xing, Jingxuan Zhu, Zekun Jiang, Wei Wang, Ziming Zhao, Yingxiu Zhang, Ming-Liang Song, Jun Jiang, Yuning Zheng, Bo |
| author_facet | Gu, Jihao Ai, Qihang Wang, Yingyao Bu, Pi Xing, Jingxuan Zhu, Zekun Jiang, Wei Wang, Ziming Zhao, Yingxiu Zhang, Ming-Liang Song, Jun Jiang, Yuning Zheng, Bo |
| contents | Vision-language model-based mobile agents have gained the ability to understand complex instructions and mobile screenshots, benefiting from reinforcement learning paradigms like Group Relative Policy Optimization (GRPO). However, existing approaches centers on offline training or local action-level rewards often trap agents in local optima, hindering effective exploration and error correction with the environment. Crucially, we find that directly applying task-level rewards often leads to convergence difficulties due to the sparse nature of GUI interactions. To address these challenges, we present \textbf{Mobile-R1}, a systematic training recipe that bridges atomic action execution and strategic task completion. We propose a hierarchical curriculum consisting of three stages: (1) format alignment for reasoning structure, (2) on-policy exploration with verifiable action feedback to ground basic execution, and (3) multi-turn task-level training with realistic environment to unlock exploration and self-correction. This hierarchical strategy effectively bootstraps the agent, significantly enhancing its capability for exploration and self-correction (the ``Eureka'' moments). Furthermore, addressing the critical scarcity of diverse GUI data in non-English ecosystems, we contribute a comprehensive Chinese mobile dataset covering 28 applications with 24,521 high-quality manual annotations, and establish a rigorous benchmark with 500 trajectories. We will open source all resources, including the dataset, benchmark, model weight, and codes: https://mobile-r1.github.io/Mobile-R1/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20332 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training Gu, Jihao Ai, Qihang Wang, Yingyao Bu, Pi Xing, Jingxuan Zhu, Zekun Jiang, Wei Wang, Ziming Zhao, Yingxiu Zhang, Ming-Liang Song, Jun Jiang, Yuning Zheng, Bo Artificial Intelligence Vision-language model-based mobile agents have gained the ability to understand complex instructions and mobile screenshots, benefiting from reinforcement learning paradigms like Group Relative Policy Optimization (GRPO). However, existing approaches centers on offline training or local action-level rewards often trap agents in local optima, hindering effective exploration and error correction with the environment. Crucially, we find that directly applying task-level rewards often leads to convergence difficulties due to the sparse nature of GUI interactions. To address these challenges, we present \textbf{Mobile-R1}, a systematic training recipe that bridges atomic action execution and strategic task completion. We propose a hierarchical curriculum consisting of three stages: (1) format alignment for reasoning structure, (2) on-policy exploration with verifiable action feedback to ground basic execution, and (3) multi-turn task-level training with realistic environment to unlock exploration and self-correction. This hierarchical strategy effectively bootstraps the agent, significantly enhancing its capability for exploration and self-correction (the ``Eureka'' moments). Furthermore, addressing the critical scarcity of diverse GUI data in non-English ecosystems, we contribute a comprehensive Chinese mobile dataset covering 28 applications with 24,521 high-quality manual annotations, and establish a rigorous benchmark with 500 trajectories. We will open source all resources, including the dataset, benchmark, model weight, and codes: https://mobile-r1.github.io/Mobile-R1/. |
| title | Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.20332 |