AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912869057036288 |
|---|---|
| author | Barke, Shraddha Goyal, Arnav Khare, Alind Singh, Avaljot Nath, Suman Bansal, Chetan |
| author_facet | Barke, Shraddha Goyal, Arnav Khare, Alind Singh, Avaljot Nath, Suman Bansal, Chetan |
| contents | AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 115 failed trajectories spanning structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AGENTRX, an automated domain-agnostic diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. Our framework improves step localization and failure attribution over existing baselines across three domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02475 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AgentRx: Diagnosing AI Agent Failures from Execution Trajectories Barke, Shraddha Goyal, Arnav Khare, Alind Singh, Avaljot Nath, Suman Bansal, Chetan Artificial Intelligence AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 115 failed trajectories spanning structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AGENTRX, an automated domain-agnostic diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. Our framework improves step localization and failure attribution over existing baselines across three domains. |
| title | AgentRx: Diagnosing AI Agent Failures from Execution Trajectories |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.02475 |