Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.02475 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912869057036288 |
|---|---|
| author | Barke, Shraddha Goyal, Arnav Khare, Alind Singh, Avaljot Nath, Suman Bansal, Chetan |
| author_facet | Barke, Shraddha Goyal, Arnav Khare, Alind Singh, Avaljot Nath, Suman Bansal, Chetan |
| contents | AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 115 failed trajectories spanning structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AGENTRX, an automated domain-agnostic diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. Our framework improves step localization and failure attribution over existing baselines across three domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02475 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AgentRx: Diagnosing AI Agent Failures from Execution Trajectories Barke, Shraddha Goyal, Arnav Khare, Alind Singh, Avaljot Nath, Suman Bansal, Chetan Artificial Intelligence AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 115 failed trajectories spanning structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AGENTRX, an automated domain-agnostic diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. Our framework improves step localization and failure attribution over existing baselines across three domains. |
| title | AgentRx: Diagnosing AI Agent Failures from Execution Trajectories |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.02475 |