GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916957453811712 |
|---|---|
| author | Ye, Xianhang Li, Yiqing Dai, Wei Liu, Miancan Chen, Ziyuan Han, Zhangye Min, Hongbo Ren, Jinkui Zhang, Xiantao Yang, Wen Jin, Zhi |
| author_facet | Ye, Xianhang Li, Yiqing Dai, Wei Liu, Miancan Chen, Ziyuan Han, Zhangye Min, Hongbo Ren, Jinkui Zhang, Xiantao Yang, Wen Jin, Zhi |
| contents | Existing GUI grounding methods often struggle with fine-grained localization in high-resolution screenshots. To address this, we propose GUI-ARP, a novel framework that enables adaptive multi-stage inference. Equipped with the proposed Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC), GUI-ARP dynamically exploits visual attention for cropping task-relevant regions and adapts its inference strategy, performing a single-stage inference for simple cases and a multi-stage analysis for more complex scenarios. This is achieved through a two-phase training pipeline that integrates supervised fine-tuning with reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate that the proposed GUI-ARP achieves state-of-the-art performance on challenging GUI grounding benchmarks, with a 7B model reaching 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark. Notably, GUI-ARP-7B demonstrates strong competitiveness against open-source 72B models (UI-TARS-72B at 38.1%) and proprietary models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15532 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents Ye, Xianhang Li, Yiqing Dai, Wei Liu, Miancan Chen, Ziyuan Han, Zhangye Min, Hongbo Ren, Jinkui Zhang, Xiantao Yang, Wen Jin, Zhi Computer Vision and Pattern Recognition Artificial Intelligence Existing GUI grounding methods often struggle with fine-grained localization in high-resolution screenshots. To address this, we propose GUI-ARP, a novel framework that enables adaptive multi-stage inference. Equipped with the proposed Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC), GUI-ARP dynamically exploits visual attention for cropping task-relevant regions and adapts its inference strategy, performing a single-stage inference for simple cases and a multi-stage analysis for more complex scenarios. This is achieved through a two-phase training pipeline that integrates supervised fine-tuning with reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate that the proposed GUI-ARP achieves state-of-the-art performance on challenging GUI grounding benchmarks, with a 7B model reaching 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark. Notably, GUI-ARP-7B demonstrates strong competitiveness against open-source 72B models (UI-TARS-72B at 38.1%) and proprietary models. |
| title | GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2509.15532 |