UI-Venus Technical Report: Building High-performance UI Agents with RFT
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915447288365056 |
|---|---|
| author | Gu, Zhangxuan Zeng, Zhengwen Xu, Zhenyu Zhou, Xingran Shen, Shuheng Liu, Yunfei Zhou, Beitong Meng, Changhua Xia, Tianyu Chen, Weizhi Wen, Yue Dou, Jingya Tang, Fei Lin, Jinzhen Liu, Yulin Guo, Zhenlin Gong, Yichen Jia, Heng Gao, Changlong Guo, Yuan Deng, Yong Guo, Zhenyu Chen, Liang Wang, Weiqiang |
| author_facet | Gu, Zhangxuan Zeng, Zhengwen Xu, Zhenyu Zhou, Xingran Shen, Shuheng Liu, Yunfei Zhou, Beitong Meng, Changhua Xia, Tianyu Chen, Weizhi Wen, Yue Dou, Jingya Tang, Fei Lin, Jinzhen Liu, Yulin Guo, Zhenlin Gong, Yichen Jia, Heng Gao, Changlong Guo, Yuan Deng, Yong Guo, Zhenyu Chen, Liang Wang, Weiqiang |
| contents | We present UI-Venus, a native UI agent that takes only screenshots as input based on a multimodal large language model. UI-Venus achieves SOTA performance on both UI grounding and navigation tasks using only several hundred thousand high-quality training samples through reinforcement finetune (RFT) based on Qwen2.5-VL. Specifically, the 7B and 72B variants of UI-Venus obtain 94.1% / 50.8% and 95.3% / 61.9% on the standard grounding benchmarks, i.e., Screenspot-V2 / Pro, surpassing the previous SOTA baselines including open-source GTA1 and closed-source UI-TARS-1.5. To show UI-Venus's summary and planing ability, we also evaluate it on the AndroidWorld, an online UI navigation arena, on which our 7B and 72B variants achieve 49.1% and 65.9% success rate, also beating existing models. To achieve this, we introduce carefully designed reward functions for both UI grounding and navigation tasks and corresponding efficient data cleaning strategies. To further boost navigation performance, we propose Self-Evolving Trajectory History Alignment & Sparse Action Enhancement that refine historical reasoning traces and balances the distribution of sparse but critical actions, leading to more coherent planning and better generalization in complex UI tasks. Our contributions include the publish of SOTA open-source UI agents, comprehensive data cleaning protocols and a novel self-evolving framework for improving navigation performance, which encourage further research and development in the community. Code is available at https://github.com/inclusionAI/UI-Venus. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_10833 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UI-Venus Technical Report: Building High-performance UI Agents with RFT Gu, Zhangxuan Zeng, Zhengwen Xu, Zhenyu Zhou, Xingran Shen, Shuheng Liu, Yunfei Zhou, Beitong Meng, Changhua Xia, Tianyu Chen, Weizhi Wen, Yue Dou, Jingya Tang, Fei Lin, Jinzhen Liu, Yulin Guo, Zhenlin Gong, Yichen Jia, Heng Gao, Changlong Guo, Yuan Deng, Yong Guo, Zhenyu Chen, Liang Wang, Weiqiang Computer Vision and Pattern Recognition We present UI-Venus, a native UI agent that takes only screenshots as input based on a multimodal large language model. UI-Venus achieves SOTA performance on both UI grounding and navigation tasks using only several hundred thousand high-quality training samples through reinforcement finetune (RFT) based on Qwen2.5-VL. Specifically, the 7B and 72B variants of UI-Venus obtain 94.1% / 50.8% and 95.3% / 61.9% on the standard grounding benchmarks, i.e., Screenspot-V2 / Pro, surpassing the previous SOTA baselines including open-source GTA1 and closed-source UI-TARS-1.5. To show UI-Venus's summary and planing ability, we also evaluate it on the AndroidWorld, an online UI navigation arena, on which our 7B and 72B variants achieve 49.1% and 65.9% success rate, also beating existing models. To achieve this, we introduce carefully designed reward functions for both UI grounding and navigation tasks and corresponding efficient data cleaning strategies. To further boost navigation performance, we propose Self-Evolving Trajectory History Alignment & Sparse Action Enhancement that refine historical reasoning traces and balances the distribution of sparse but critical actions, leading to more coherent planning and better generalization in complex UI tasks. Our contributions include the publish of SOTA open-source UI agents, comprehensive data cleaning protocols and a novel self-evolving framework for improving navigation performance, which encourage further research and development in the community. Code is available at https://github.com/inclusionAI/UI-Venus. |
| title | UI-Venus Technical Report: Building High-performance UI Agents with RFT |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.10833 |