Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2510.19205 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911226282377216 |
|---|---|
| author | Qian, Yaoyao Wang, Yuanli Zhang, Jinda Zong, Yun Chen, Meixu Zhou, Hanhan Huang, Jindan Zeng, Yifan Hu, Xinyu Song, Chan Hee Zhang, Danqing |
| author_facet | Qian, Yaoyao Wang, Yuanli Zhang, Jinda Zong, Yun Chen, Meixu Zhou, Hanhan Huang, Jindan Zeng, Yifan Hu, Xinyu Song, Chan Hee Zhang, Danqing |
| contents | Current evaluation of web agents largely reduces to binary success metrics or conformity to a single reference trajectory, ignoring the structural diversity present in benchmark datasets. We present WebGraphEval, a framework that abstracts trajectories from multiple agents into a unified, weighted action graph. This representation is directly compatible with benchmarks such as WebArena, leveraging leaderboard runs and newly collected trajectories without modifying environments. The framework canonically encodes actions, merges recurring behaviors, and applies structural analyses including reward propagation and success-weighted edge statistics. Evaluations across thousands of trajectories from six web agents show that the graph abstraction captures cross-model regularities, highlights redundancy and inefficiency, and identifies critical decision points overlooked by outcome-based metrics. By framing web interaction as graph-structured data, WebGraphEval establishes a general methodology for multi-path, cross-agent, and efficiency-aware evaluation of web agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_19205 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation Qian, Yaoyao Wang, Yuanli Zhang, Jinda Zong, Yun Chen, Meixu Zhou, Hanhan Huang, Jindan Zeng, Yifan Hu, Xinyu Song, Chan Hee Zhang, Danqing Artificial Intelligence Current evaluation of web agents largely reduces to binary success metrics or conformity to a single reference trajectory, ignoring the structural diversity present in benchmark datasets. We present WebGraphEval, a framework that abstracts trajectories from multiple agents into a unified, weighted action graph. This representation is directly compatible with benchmarks such as WebArena, leveraging leaderboard runs and newly collected trajectories without modifying environments. The framework canonically encodes actions, merges recurring behaviors, and applies structural analyses including reward propagation and success-weighted edge statistics. Evaluations across thousands of trajectories from six web agents show that the graph abstraction captures cross-model regularities, highlights redundancy and inefficiency, and identifies critical decision points overlooked by outcome-based metrics. By framing web interaction as graph-structured data, WebGraphEval establishes a general methodology for multi-path, cross-agent, and efficiency-aware evaluation of web agents. |
| title | WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.19205 |