Table of Contents: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Yan, Haolong, Wang, Jia, Huang, Xin, Shen, Yeqing, Meng, Ziyang, Fan, Zhimin, Tan, Kaijun, Gao, Jin, Shi, Lieyu, Yang, Mi, Yang, Shiliang, Wang, Zhirui, Li, Brian, An, Kang, Li, Chenyang, Lei, Lei, Duan, Mengmeng, Liang, Danxun, Liu, Guodong, Cheng, Hang, Wu, Hao, Dong, Jie, Huang, Junhao, Chen, Mei, Yu, Renjie, Li, Shunshan, Zhou, Xu, Dai, Yiting, Deng, Yineng, Liang, Yingdan, Chen, Zelin, Sun, Wen, Yan, Chengxu, Xu, Chunqin, Li, Dong, Xiao, Fengqiong, Fan, Guanghao, Li, Guopeng, Peng, Guozhen, Li, Hongbing, Li, Hang, Chen, Hongming, Xie, Jingjing, Li, Jianyong, Zhang, Jingyang, Ren, Jiaju, Yuan, Jiayu, Yin, Jianpeng, Cao, Kai, Zhao, Liang, Tan, Liguo, Shi, Liying, Ren, Mengqiang, Xu, Min, Liu, Manjiao, Luo, Mao, Wan, Mingxin, Wang, Na, Wu, Nan, Wang, Ning, Ma, Peiyao, Zhang, Qingzhou, Wang, Qiao, Zeng, Qinlin, Gao, Qiong, Li, Qiongyao, Zhong, Shangwu, Gao, Shuli, Liu, Shaofan, Gao, Shisi, Luo, Shuang, Liu, Xingbin, Liu, Xiaojia, Hou, Xiaojie, Liu, Xin, Feng, Xuanti, Cai, Xuedan, Wen, Xuan, Zhu, Xianwei, Liang, Xin, Zhou, Xin, Sui, Yifan, Zhao, Yingxiu, Shi, Yukang, Xu, Yunfang, Zeng, Yuqing, Zhang, Yixun, Weng, Zejia, Yan, Zhonghao, Huang, Zhiguo, Wang, Zhuoyu, Yan, Zihan, Ge, Zheng, Li, Jing, Zhu, Yibo, Jiao, Binxing, Zhang, Xiangyu, Jiang, Daxin
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition
Online Access:	https://arxiv.org/abs/2512.15431
Tags:	Add Tag No Tags, Be the first to tag this record!

Table of Contents:

Recent advances in multimodal large language models unlock unprecedented opportunities for GUI automation. However, a fundamental challenge remains: how to efficiently acquire high-quality training data while maintaining annotation reliability? We introduce a self-evolving training pipeline powered by the Calibrated Step Reward System, which converts model-generated trajectories into reliable training signals through trajectory-level calibration, achieving >90% annotation accuracy with 10-100x lower cost. Leveraging this pipeline, we introduce Step-GUI, a family of models (4B/8B) that achieves state-of-the-art GUI performance (8B: 80.2% AndroidWorld, 48.5% OSWorld, 62.6% ScreenShot-Pro) while maintaining robust general capabilities. As GUI agent capabilities improve, practical deployment demands standardized interfaces across heterogeneous devices while protecting user privacy. To this end, we propose GUI-MCP, the first Model Context Protocol for GUI automation with hierarchical architecture that combines low-level atomic operations and high-level task delegation to local specialist models, enabling high-privacy execution where sensitive data stays on-device. Finally, to assess whether agents can handle authentic everyday usage, we introduce AndroidDaily, a benchmark grounded in real-world mobile usage patterns with 3146 static actions and 235 end-to-end tasks across high-frequency daily scenarios (8B: static 89.91%, end-to-end 52.50%). Our work advances the development of practical GUI agents and demonstrates strong potential for real-world deployment in everyday digital interactions.

Similar Items