Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.14014 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912767612551168 |
|---|---|
| author | Li, Shufan Kallidromitis, Konstantinos Gokul, Akash Kato, Yusuke Kozuka, Kazuki Grover, Aditya |
| author_facet | Li, Shufan Kallidromitis, Konstantinos Gokul, Akash Kato, Yusuke Kozuka, Kazuki Grover, Aditya |
| contents | World models have shown great utility in improving the task performance of embodied agents. While prior work largely focuses on pixel-space world models, these approaches face practical limitations in GUI settings, where predicting complex visual elements in future states is often difficult. In this work, we explore an alternative formulation of world modeling for GUI agents, where state transitions are described in natural language rather than predicting raw pixels. First, we introduce MobileWorldBench, a benchmark that evaluates the ability of vision-language models (VLMs) to function as world models for mobile GUI agents. Second, we release MobileWorld, a large-scale dataset consisting of 1.4M samples, that significantly improves the world modeling capabilities of VLMs. Finally, we propose a novel framework that integrates VLM world models into the planning framework of mobile agents, demonstrating that semantic world models can directly benefit mobile agents by improving task success rates. The code and dataset is available at https://github.com/jacklishufan/MobileWorld |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14014 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MobileWorldBench: Towards Semantic World Modeling For Mobile Agents Li, Shufan Kallidromitis, Konstantinos Gokul, Akash Kato, Yusuke Kozuka, Kazuki Grover, Aditya Artificial Intelligence World models have shown great utility in improving the task performance of embodied agents. While prior work largely focuses on pixel-space world models, these approaches face practical limitations in GUI settings, where predicting complex visual elements in future states is often difficult. In this work, we explore an alternative formulation of world modeling for GUI agents, where state transitions are described in natural language rather than predicting raw pixels. First, we introduce MobileWorldBench, a benchmark that evaluates the ability of vision-language models (VLMs) to function as world models for mobile GUI agents. Second, we release MobileWorld, a large-scale dataset consisting of 1.4M samples, that significantly improves the world modeling capabilities of VLMs. Finally, we propose a novel framework that integrates VLM world models into the planning framework of mobile agents, demonstrating that semantic world models can directly benefit mobile agents by improving task success rates. The code and dataset is available at https://github.com/jacklishufan/MobileWorld |
| title | MobileWorldBench: Towards Semantic World Modeling For Mobile Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.14014 |