Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.00676 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917302198337536 |
|---|---|
| author | Wu, Zhe Mo, Donglin Lu, Hongjin Xing, Junliang Liu, Jianheng Jing, Yuheng Li, Kai Shao, Kun Hao, Jianye Shi, Yuanchun |
| author_facet | Wu, Zhe Mo, Donglin Lu, Hongjin Xing, Junliang Liu, Jianheng Jing, Yuheng Li, Kai Shao, Kun Hao, Jianye Shi, Yuanchun |
| contents | Existing mobile device control agents often perform poorly when solving complex tasks requiring long-horizon planning and precise operations, typically due to a lack of relevant task experience or unfamiliarity with skill execution. We propose K2-Agent, a hierarchical framework that models human-like cognition by separating and co-evolving declarative (knowing what) and procedural (knowing how) knowledge for planning and execution. K2-Agent's high level reasoner is bootstrapped from a single demonstration per task and runs a Summarize-Reflect-Locate-Revise (SRLR) loop to distill and iteratively refine task-level declarative knowledge through self-evolution. The low-level executor is trained with our curriculum-guided Group Relative Policy Optimization (C-GRPO), which (i) constructs a balanced sample pool using decoupled reward signals and (ii) employs dynamic demonstration injection to guide the model in autonomously generating successful trajectories for training. On the challenging AndroidWorld benchmark, K2-Agent achieves a 76.1% success rate using only raw screenshots and open-source backbones. Furthermore, K2-Agent shows powerful dual generalization: its high-level declarative knowledge transfers across diverse base models, while its low-level procedural skills achieve competitive performance on unseen tasks in ScreenSpot-v2 and Android-in-the-Wild (AitW). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_00676 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | K^2-Agent: Co-Evolving Know-What and Know-How for Hierarchical Mobile Device Control Wu, Zhe Mo, Donglin Lu, Hongjin Xing, Junliang Liu, Jianheng Jing, Yuheng Li, Kai Shao, Kun Hao, Jianye Shi, Yuanchun Artificial Intelligence Existing mobile device control agents often perform poorly when solving complex tasks requiring long-horizon planning and precise operations, typically due to a lack of relevant task experience or unfamiliarity with skill execution. We propose K2-Agent, a hierarchical framework that models human-like cognition by separating and co-evolving declarative (knowing what) and procedural (knowing how) knowledge for planning and execution. K2-Agent's high level reasoner is bootstrapped from a single demonstration per task and runs a Summarize-Reflect-Locate-Revise (SRLR) loop to distill and iteratively refine task-level declarative knowledge through self-evolution. The low-level executor is trained with our curriculum-guided Group Relative Policy Optimization (C-GRPO), which (i) constructs a balanced sample pool using decoupled reward signals and (ii) employs dynamic demonstration injection to guide the model in autonomously generating successful trajectories for training. On the challenging AndroidWorld benchmark, K2-Agent achieves a 76.1% success rate using only raw screenshots and open-source backbones. Furthermore, K2-Agent shows powerful dual generalization: its high-level declarative knowledge transfers across diverse base models, while its low-level procedural skills achieve competitive performance on unseen tasks in ScreenSpot-v2 and Android-in-the-Wild (AitW). |
| title | K^2-Agent: Co-Evolving Know-What and Know-How for Hierarchical Mobile Device Control |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.00676 |