OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911369878568960 |
|---|---|
| author | Yang, Bowen Jin, Kaiming Wu, Zhenyu Liu, Zhaoyang Sun, Qiushi Li, Zehao Xie, JingJing Liu, Zhoumianze Xu, Fangzhi Cheng, Kanzhi Li, Qingyun Wang, Yian Qiao, Yu Wang, Zun Ding, Zichen |
| author_facet | Yang, Bowen Jin, Kaiming Wu, Zhenyu Liu, Zhaoyang Sun, Qiushi Li, Zehao Xie, JingJing Liu, Zhoumianze Xu, Fangzhi Cheng, Kanzhi Li, Qingyun Wang, Yian Qiao, Yu Wang, Zun Ding, Zichen |
| contents | While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_07779 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent Yang, Bowen Jin, Kaiming Wu, Zhenyu Liu, Zhaoyang Sun, Qiushi Li, Zehao Xie, JingJing Liu, Zhoumianze Xu, Fangzhi Cheng, Kanzhi Li, Qingyun Wang, Yian Qiao, Yu Wang, Zun Ding, Zichen Multiagent Systems Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Human-Computer Interaction I.2.10 While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld. |
| title | OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent |
| topic | Multiagent Systems Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Human-Computer Interaction I.2.10 |
| url | https://arxiv.org/abs/2601.07779 |