SecAgent: Efficient Mobile GUI Agent with Semantic Context
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910087076904960 |
|---|---|
| author | Xie, Yiping Chen, Song Xing, Jingxuan Jiang, Wei Zhu, Zekun Wang, Yingyao Bu, Pi Song, Jun Jiang, Yuning Zheng, Bo |
| author_facet | Xie, Yiping Chen, Song Xing, Jingxuan Jiang, Wei Zhu, Zekun Wang, Yingyao Bu, Pi Song, Jun Jiang, Yuning Zheng, Bo |
| contents | Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the scarcity of high-quality multilingual datasets, particularly for non-English ecosystems, and inefficient history representation methods. To address these challenges, we present SecAgent, an efficient mobile GUI agent at 3B scale. We first construct a human-verified Chinese mobile GUI dataset with 18k grounding samples and 121k navigation steps across 44 applications, along with a Chinese navigation benchmark featuring multi-choice action annotations. Building upon this dataset, we propose a semantic context mechanism that distills history screenshots and actions into concise, natural language summaries, significantly reducing computational costs while preserving task-relevant information. Through supervised and reinforcement fine-tuning, SecAgent outperforms similar-scale baselines and achieves performance comparable to 7B-8B models on our and public navigation benchmarks. Our dataset is available at https://huggingface.co/datasets/alibabagroup/CMGUI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08533 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SecAgent: Efficient Mobile GUI Agent with Semantic Context Xie, Yiping Chen, Song Xing, Jingxuan Jiang, Wei Zhu, Zekun Wang, Yingyao Bu, Pi Song, Jun Jiang, Yuning Zheng, Bo Computer Vision and Pattern Recognition Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the scarcity of high-quality multilingual datasets, particularly for non-English ecosystems, and inefficient history representation methods. To address these challenges, we present SecAgent, an efficient mobile GUI agent at 3B scale. We first construct a human-verified Chinese mobile GUI dataset with 18k grounding samples and 121k navigation steps across 44 applications, along with a Chinese navigation benchmark featuring multi-choice action annotations. Building upon this dataset, we propose a semantic context mechanism that distills history screenshots and actions into concise, natural language summaries, significantly reducing computational costs while preserving task-relevant information. Through supervised and reinforcement fine-tuning, SecAgent outperforms similar-scale baselines and achieves performance comparable to 7B-8B models on our and public navigation benchmarks. Our dataset is available at https://huggingface.co/datasets/alibabagroup/CMGUI. |
| title | SecAgent: Efficient Mobile GUI Agent with Semantic Context |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.08533 |