Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.20650 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911415967678464 |
|---|---|
| author | Wu, Jing Barretto, Daphne Chen, Yiye Gydé, Nicholas Jian, Yanan He, Yuhang Vineet, Vibhav |
| author_facet | Wu, Jing Barretto, Daphne Chen, Yiye Gydé, Nicholas Jian, Yanan He, Yuhang Vineet, Vibhav |
| contents | Long-horizon, repetitive workflows are common in professional settings, such as processing expense reports from receipts and entering student grades from exam papers. These tasks are often tedious for humans since they can extend to extreme lengths proportional to the size of the data to process. However, they are ideal for Computer-Use Agents (CUAs) due to their structured, recurring sub-workflows with logic that can be systematically learned. Identifying the absence of an evaluation benchmark as a primary bottleneck, we establish OS-Marathon, comprising 242 long-horizon, repetitive tasks across 2 domains to evaluate state-of-the-art (SOTA) agents. We then introduce a cost-effective method to construct a condensed demonstration using only few-shot examples to teach agents the underlying workflow logic, enabling them to execute similar workflows effectively on larger, unseen data collections. Extensive experiments demonstrate both the inherent challenges of these tasks and the effectiveness of our proposed method. Project website: https://os-marathon.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20650 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks Wu, Jing Barretto, Daphne Chen, Yiye Gydé, Nicholas Jian, Yanan He, Yuhang Vineet, Vibhav Computer Vision and Pattern Recognition Long-horizon, repetitive workflows are common in professional settings, such as processing expense reports from receipts and entering student grades from exam papers. These tasks are often tedious for humans since they can extend to extreme lengths proportional to the size of the data to process. However, they are ideal for Computer-Use Agents (CUAs) due to their structured, recurring sub-workflows with logic that can be systematically learned. Identifying the absence of an evaluation benchmark as a primary bottleneck, we establish OS-Marathon, comprising 242 long-horizon, repetitive tasks across 2 domains to evaluate state-of-the-art (SOTA) agents. We then introduce a cost-effective method to construct a condensed demonstration using only few-shot examples to teach agents the underlying workflow logic, enabling them to execute similar workflows effectively on larger, unseen data collections. Extensive experiments demonstrate both the inherent challenges of these tasks and the effectiveness of our proposed method. Project website: https://os-marathon.github.io/. |
| title | OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.20650 |