UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915911631372288 |
|---|---|
| author | Wu, Mengzhou Guo, Yuzhe Cao, Yuan Lu, Haochuan Zhu, Songhe Qu, Pingzhe Chen, Xin Qin, Kang Wang, Zhongpu Zhang, Xiaode Wang, Xinyi Dai, Wei Cao, Gang Deng, Yuetang Gong, Zhi Ran, Dezhi Li, Linyi Yang, Wei Xie, Tao |
| author_facet | Wu, Mengzhou Guo, Yuzhe Cao, Yuan Lu, Haochuan Zhu, Songhe Qu, Pingzhe Chen, Xin Qin, Kang Wang, Zhongpu Zhang, Xiaode Wang, Xinyi Dai, Wei Cao, Gang Deng, Yuetang Gong, Zhi Ran, Dezhi Li, Linyi Yang, Wei Xie, Tao |
| contents | Scaling generalist GUI agents is hindered by the data scalability bottleneck of expensive human demonstrations and the "distillation ceiling" of synthetic teacher supervision. To transcend these limitations, we propose UI-Oceanus, a framework that shifts the learning focus from mimicking high-level trajectories to mastering interaction physics via ground-truth environmental feedback. Through a systematic investigation of self-supervised objectives, we identify that forward dynamics, defined as the generative prediction of future interface states, acts as the primary driver for scalability and significantly outweighs inverse inference. UI-Oceanus leverages this insight by converting low-cost autonomous exploration, which is verified directly by system execution, into high-density generative supervision to construct a robust internal world model. Experimental evaluations across a series of models demonstrate the decisive superiority of our approach: models utilizing Continual Pre-Training (CPT) on synthetic dynamics outperform non-CPT baselines with an average success rate improvement of 7% on offline benchmarks, which amplifies to a 16.8% gain in real-world online navigation. Furthermore, we observe that navigation performance scales with synthetic data volume. These results confirm that grounding agents in forward predictive modeling offers a superior pathway to scalable GUI automation with robust cross-domain adaptability and compositional generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_02345 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics Wu, Mengzhou Guo, Yuzhe Cao, Yuan Lu, Haochuan Zhu, Songhe Qu, Pingzhe Chen, Xin Qin, Kang Wang, Zhongpu Zhang, Xiaode Wang, Xinyi Dai, Wei Cao, Gang Deng, Yuetang Gong, Zhi Ran, Dezhi Li, Linyi Yang, Wei Xie, Tao Machine Learning Artificial Intelligence Scaling generalist GUI agents is hindered by the data scalability bottleneck of expensive human demonstrations and the "distillation ceiling" of synthetic teacher supervision. To transcend these limitations, we propose UI-Oceanus, a framework that shifts the learning focus from mimicking high-level trajectories to mastering interaction physics via ground-truth environmental feedback. Through a systematic investigation of self-supervised objectives, we identify that forward dynamics, defined as the generative prediction of future interface states, acts as the primary driver for scalability and significantly outweighs inverse inference. UI-Oceanus leverages this insight by converting low-cost autonomous exploration, which is verified directly by system execution, into high-density generative supervision to construct a robust internal world model. Experimental evaluations across a series of models demonstrate the decisive superiority of our approach: models utilizing Continual Pre-Training (CPT) on synthetic dynamics outperform non-CPT baselines with an average success rate improvement of 7% on offline benchmarks, which amplifies to a 16.8% gain in real-world online navigation. Furthermore, we observe that navigation performance scales with synthetic data volume. These results confirm that grounding agents in forward predictive modeling offers a superior pathway to scalable GUI automation with robust cross-domain adaptability and compositional generalization. |
| title | UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.02345 |