Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2603.01357 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911476575371264 |
|---|---|
| author | Xiu, Zidi Sun, David Q. Cheng, Kevin Patel, Maitrik Date, Josh Zhang, Yizhe Lu, Jiarui Attia, Omar Vemulapalli, Raviteja Tuzel, Oncel Cao, Meng Bengio, Samy |
| author_facet | Xiu, Zidi Sun, David Q. Cheng, Kevin Patel, Maitrik Date, Josh Zhang, Yizhe Lu, Jiarui Attia, Omar Vemulapalli, Raviteja Tuzel, Oncel Cao, Meng Bengio, Samy |
| contents | Next-generation AI must manage vast personal data, diverse tools, and multi-step reasoning, yet most benchmarks remain context-free and single-turn. We present ASTRA-bench (Assistant Skills in Tool-use, Reasoning \& Action-planning), a benchmark that uniquely unifies time-evolving personal context with an interactive toolbox and complex user intents. Our event-driven pipeline generates 2,413 scenarios across four protagonists, grounded in longitudinal life events and annotated by referential, functional, and informational complexity. Evaluation of state-of-the-art models (e.g., Claude-4.5-Opus, DeepSeek-V3.2) reveals significant performance degradation under high-complexity conditions, with argument generation emerging as the primary bottleneck. These findings expose critical limitations in current agents' ability to ground reasoning within messy personal context and orchestrate reliable multi-step plans. We release ASTRA-bench with a full execution environment and evaluation scripts to provide a diagnostic testbed for developing truly context-aware AI assistants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01357 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Context Xiu, Zidi Sun, David Q. Cheng, Kevin Patel, Maitrik Date, Josh Zhang, Yizhe Lu, Jiarui Attia, Omar Vemulapalli, Raviteja Tuzel, Oncel Cao, Meng Bengio, Samy Artificial Intelligence Next-generation AI must manage vast personal data, diverse tools, and multi-step reasoning, yet most benchmarks remain context-free and single-turn. We present ASTRA-bench (Assistant Skills in Tool-use, Reasoning \& Action-planning), a benchmark that uniquely unifies time-evolving personal context with an interactive toolbox and complex user intents. Our event-driven pipeline generates 2,413 scenarios across four protagonists, grounded in longitudinal life events and annotated by referential, functional, and informational complexity. Evaluation of state-of-the-art models (e.g., Claude-4.5-Opus, DeepSeek-V3.2) reveals significant performance degradation under high-complexity conditions, with argument generation emerging as the primary bottleneck. These findings expose critical limitations in current agents' ability to ground reasoning within messy personal context and orchestrate reliable multi-step plans. We release ASTRA-bench with a full execution environment and evaluation scripts to provide a diagnostic testbed for developing truly context-aware AI assistants. |
| title | ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Context |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.01357 |