The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908761405259776 |
|---|---|
| author | Fu, Daocheng Mei, Jianbiao Wu, Rong Yang, Xuemeng Xu, Jia Wang, Ding Cai, Pinlong Liu, Yong Wen, Licheng Shi, Botian |
| author_facet | Fu, Daocheng Mei, Jianbiao Wu, Rong Yang, Xuemeng Xu, Jia Wang, Ding Cai, Pinlong Liu, Yong Wen, Licheng Shi, Botian |
| contents | The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static environments, overlooking robustness for stochastic real-world deployment. We identify three key challenges: dynamic task scheduling, active exploration under uncertainty, and continuous learning from experience. To bridge this gap, we introduce \method{}, a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. Unlike traditional benchmarks, \method{} evaluates agents along three dimensions: (1) context-aware scheduling for streaming tasks with varying priorities; (2) prudent information acquisition to reduce hallucination via active exploration; and (3) continuous evolution by distilling generalized strategies from rule-based, dynamically generated tasks. Experiments show that cutting-edge agents have significant deficiencies in dynamic environments, especially in active exploration and continual learning. Our work establishes a framework for assessing agent reliability, shifting evaluation from static tests to realistic, production-oriented scenarios. Our codes are available at https://github.com/KnowledgeXLab/EvoEnv |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_08173 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios Fu, Daocheng Mei, Jianbiao Wu, Rong Yang, Xuemeng Xu, Jia Wang, Ding Cai, Pinlong Liu, Yong Wen, Licheng Shi, Botian Artificial Intelligence The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static environments, overlooking robustness for stochastic real-world deployment. We identify three key challenges: dynamic task scheduling, active exploration under uncertainty, and continuous learning from experience. To bridge this gap, we introduce \method{}, a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. Unlike traditional benchmarks, \method{} evaluates agents along three dimensions: (1) context-aware scheduling for streaming tasks with varying priorities; (2) prudent information acquisition to reduce hallucination via active exploration; and (3) continuous evolution by distilling generalized strategies from rule-based, dynamically generated tasks. Experiments show that cutting-edge agents have significant deficiencies in dynamic environments, especially in active exploration and continual learning. Our work establishes a framework for assessing agent reliability, shifting evaluation from static tests to realistic, production-oriented scenarios. Our codes are available at https://github.com/KnowledgeXLab/EvoEnv |
| title | The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2601.08173 |