Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909042817892352 |
|---|---|
| author | Tang, Zirui Zhou, Xuanhe Liu, Yumou Li, Linchun Wu, Yukai Wang, Weizheng Huang, Hongzhang Zhou, Wei Zhou, Jun Song, Jiachen Yu, Shaoli Wang, Jinqi Zhou, Zihang Zhou, Hongyi Lv, Yuting Li, Jinyang Liu, Jiashuo Chen, Ruoyu Liu, Chunwei Li, GuoLiang Kang, Jihua Wu, Fan |
| author_facet | Tang, Zirui Zhou, Xuanhe Liu, Yumou Li, Linchun Wu, Yukai Wang, Weizheng Huang, Hongzhang Zhou, Wei Zhou, Jun Song, Jiachen Yu, Shaoli Wang, Jinqi Zhou, Zihang Zhou, Hongyi Lv, Yuting Li, Jinyang Liu, Jiashuo Chen, Ruoyu Liu, Chunwei Li, GuoLiang Kang, Jihua Wu, Fan |
| contents | Workspace learning requires AI agents to identify, reason over, exploit, and update explicit and implicit dependencies among heterogeneous files in a worker's workspace, enabling them to complete both routine and advanced tasks effectively. Despite its importance, existing relevant benchmarks largely evaluate agents on pre-specified or synthesized files with limited real-world dependencies, leaving workspace-level evaluation underexplored. To this end, we introduce Workspace-Bench, a benchmark for evaluating AI agents on Workspace Learning involving Large-Scale File Dependencies. We construct realistic workspaces with 5 worker profiles, 74 file types, 20,476 files (up to 20GB) and curate 388 tasks, each with its own file dependency graph, evaluated across 7,399 total rubrics that require cross-file retrieval, contextual reasoning, and adaptive decision-making. We further provide Workspace-Bench-Lite, a 100-task subset that preserves the benchmark distribution while reducing evaluation costs by about 70%. We evaluate 4 popular agent harnesses and 7 foundation models. Experimental results show that current agents remain far from reliable workspace learning, where the best reaches only about 60%, substantially below the human result of 80.7%, and the average performance across agents is only 43.3%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_03596 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies Tang, Zirui Zhou, Xuanhe Liu, Yumou Li, Linchun Wu, Yukai Wang, Weizheng Huang, Hongzhang Zhou, Wei Zhou, Jun Song, Jiachen Yu, Shaoli Wang, Jinqi Zhou, Zihang Zhou, Hongyi Lv, Yuting Li, Jinyang Liu, Jiashuo Chen, Ruoyu Liu, Chunwei Li, GuoLiang Kang, Jihua Wu, Fan Artificial Intelligence Computation and Language Databases Machine Learning Workspace learning requires AI agents to identify, reason over, exploit, and update explicit and implicit dependencies among heterogeneous files in a worker's workspace, enabling them to complete both routine and advanced tasks effectively. Despite its importance, existing relevant benchmarks largely evaluate agents on pre-specified or synthesized files with limited real-world dependencies, leaving workspace-level evaluation underexplored. To this end, we introduce Workspace-Bench, a benchmark for evaluating AI agents on Workspace Learning involving Large-Scale File Dependencies. We construct realistic workspaces with 5 worker profiles, 74 file types, 20,476 files (up to 20GB) and curate 388 tasks, each with its own file dependency graph, evaluated across 7,399 total rubrics that require cross-file retrieval, contextual reasoning, and adaptive decision-making. We further provide Workspace-Bench-Lite, a 100-task subset that preserves the benchmark distribution while reducing evaluation costs by about 70%. We evaluate 4 popular agent harnesses and 7 foundation models. Experimental results show that current agents remain far from reliable workspace learning, where the best reaches only about 60%, substantially below the human result of 80.7%, and the average performance across agents is only 43.3%. |
| title | Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies |
| topic | Artificial Intelligence Computation and Language Databases Machine Learning |
| url | https://arxiv.org/abs/2605.03596 |