PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914330867400704 |
|---|---|
| author | Arman, Shifat E. Sakib, Syed Nazmus Taton, Tapodhir Karmakar Haque, Nafiul Amin, Shahrear Bin |
| author_facet | Arman, Shifat E. Sakib, Syed Nazmus Taton, Tapodhir Karmakar Haque, Nafiul Amin, Shahrear Bin |
| contents | We introduce PATHWAYS, a benchmark of 250 multi-step decision tasks that test whether web-based agents can discover and correctly use hidden contextual information. Across both closed and open models, agents typically navigate to relevant pages but retrieve decisive hidden evidence in only a small fraction of cases. When tasks require overturning misleading surface-level signals, performance drops sharply to near chance accuracy. Agents frequently hallucinate investigative reasoning by claiming to rely on evidence they never accessed. Even when correct context is discovered, agents often fail to integrate it into their final decision. Providing more explicit instructions improves context discovery but often reduces overall accuracy, revealing a tradeoff between procedural compliance and effective judgement. Together, these results show that current web agent architectures lack reliable mechanisms for adaptive investigation, evidence integration, and judgement override. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05354 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents Arman, Shifat E. Sakib, Syed Nazmus Taton, Tapodhir Karmakar Haque, Nafiul Amin, Shahrear Bin Artificial Intelligence We introduce PATHWAYS, a benchmark of 250 multi-step decision tasks that test whether web-based agents can discover and correctly use hidden contextual information. Across both closed and open models, agents typically navigate to relevant pages but retrieve decisive hidden evidence in only a small fraction of cases. When tasks require overturning misleading surface-level signals, performance drops sharply to near chance accuracy. Agents frequently hallucinate investigative reasoning by claiming to rely on evidence they never accessed. Even when correct context is discovered, agents often fail to integrate it into their final decision. Providing more explicit instructions improves context discovery but often reduces overall accuracy, revealing a tradeoff between procedural compliance and effective judgement. Together, these results show that current web agent architectures lack reliable mechanisms for adaptive investigation, evidence integration, and judgement override. |
| title | PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.05354 |