Scaling Web Agent Training through Automatic Data Generation and Fine-grained Evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908831877955584 |
|---|---|
| author | Logeswaran, Lajanugen Kim, Jaekyeom Sohn, Sungryull Glasscock, Creighton Lee, Honglak |
| author_facet | Logeswaran, Lajanugen Kim, Jaekyeom Sohn, Sungryull Glasscock, Creighton Lee, Honglak |
| contents | We present a scalable pipeline for automatically generating high-quality training data for web agents. In particular, a major challenge in identifying high-quality training instances is trajectory evaluation - quantifying how much progress was made towards task completion. We introduce a novel constraint-based evaluation framework that provides fine-grained assessment of progress towards task completion. This enables us to leverage partially successful trajectories, which significantly expands the amount of usable training data. We evaluate our method on a new benchmark we propose called BookingArena, which consists of complex booking tasks across 20 popular websites, and demonstrate that our distilled student model outperforms open-source approaches and matches or exceeds commercial systems, while being a significantly smaller model. Our work addresses the challenge of efficiently creating diverse, realistic web interaction datasets and provides a systematic evaluation methodology for complex structured web tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_12544 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Scaling Web Agent Training through Automatic Data Generation and Fine-grained Evaluation Logeswaran, Lajanugen Kim, Jaekyeom Sohn, Sungryull Glasscock, Creighton Lee, Honglak Artificial Intelligence We present a scalable pipeline for automatically generating high-quality training data for web agents. In particular, a major challenge in identifying high-quality training instances is trajectory evaluation - quantifying how much progress was made towards task completion. We introduce a novel constraint-based evaluation framework that provides fine-grained assessment of progress towards task completion. This enables us to leverage partially successful trajectories, which significantly expands the amount of usable training data. We evaluate our method on a new benchmark we propose called BookingArena, which consists of complex booking tasks across 20 popular websites, and demonstrate that our distilled student model outperforms open-source approaches and matches or exceeds commercial systems, while being a significantly smaller model. Our work addresses the challenge of efficiently creating diverse, realistic web interaction datasets and provides a systematic evaluation methodology for complex structured web tasks. |
| title | Scaling Web Agent Training through Automatic Data Generation and Fine-grained Evaluation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.12544 |