Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908780263899136 |
|---|---|
| author | Cheng, Yuhao Tang, Liang Li, Shuxian Huo, Yukang Duan, Tiaonan Huang, Kaer Jing, Yanzhe Yan, Yiqiang |
| author_facet | Cheng, Yuhao Tang, Liang Li, Shuxian Huo, Yukang Duan, Tiaonan Huang, Kaer Jing, Yanzhe Yan, Yiqiang |
| contents | Computer use agents represent an emerging area in artificial intelligence, aiming to operate computers autonomously to fulfill user tasks, attracting significant attention from both industry and academia. However, the performance of existing agents remains insufficient for practical deployment. In this paper, we propose the Self-Evolution Agent (SEA) for computer operation, alongside three core innovations in data generation, reinforcement learning, and model enhancement to develop this agent. Specifically, we first design an automatic pipeline to generate verifiable task trajectories for training. Second, we propose Efficient Step-wise Reinforcement Learning to reduce the substantial computational overhead of long-horizon training. Finally, we introduce a model enhancement method that integrates grounding and planning capabilities into a single model without additional training. Leveraging these innovations, our SEA (with only 7B parameters) outperforms existing models of the same parameter scale and achieves performance comparable to larger models (e.g., 32B/72B parameters) on computer use tasks. We plan to release the model weights and related code as open-source resources in the future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_04037 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent Cheng, Yuhao Tang, Liang Li, Shuxian Huo, Yukang Duan, Tiaonan Huang, Kaer Jing, Yanzhe Yan, Yiqiang Artificial Intelligence Computer use agents represent an emerging area in artificial intelligence, aiming to operate computers autonomously to fulfill user tasks, attracting significant attention from both industry and academia. However, the performance of existing agents remains insufficient for practical deployment. In this paper, we propose the Self-Evolution Agent (SEA) for computer operation, alongside three core innovations in data generation, reinforcement learning, and model enhancement to develop this agent. Specifically, we first design an automatic pipeline to generate verifiable task trajectories for training. Second, we propose Efficient Step-wise Reinforcement Learning to reduce the substantial computational overhead of long-horizon training. Finally, we introduce a model enhancement method that integrates grounding and planning capabilities into a single model without additional training. Leveraging these innovations, our SEA (with only 7B parameters) outperforms existing models of the same parameter scale and achieves performance comparable to larger models (e.g., 32B/72B parameters) on computer use tasks. We plan to release the model weights and related code as open-source resources in the future. |
| title | Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2508.04037 |