WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916037729976320 |
|---|---|
| author | Zhang, Wenyu You, Guoliang Tianlun Zhao, Haotian Zhu, Tianshu Wang, Haoran Tang, Xiaoxuan Dai, Mingyang Gu, Jingnan Dong, Daxiang Wu, Jianmin |
| author_facet | Zhang, Wenyu You, Guoliang Tianlun Zhao, Haotian Zhu, Tianshu Wang, Haoran Tang, Xiaoxuan Dai, Mingyang Gu, Jingnan Dong, Daxiang Wu, Jianmin |
| contents | Coding agents are increasingly used as application builders, yet many evaluations still focus on source code, repository-level tests, or intermediate traces rather than the delivered application. We introduce WebGameBench, a requirement-to-application benchmark that evaluates whether coding agents can turn a frozen Structured WebGame Specification into a browser-accessible game. Browser-native games provide a compact but behavior-dense testbed: even simple games require coordinated input handling, spatial mapping, rule execution, state transitions, terminal conditions, restart behavior, and visible feedback. In WebGameBench, each generated artifact is built, served, and exposed as a browser-accessible application under a unified deployment protocol. A runtime evaluator then interacts with the delivered game in a real browser and assigns a three-way label: EXCELLENT, USABLE, or UNUSABLE. On a human-reviewed subset, the runtime label is broadly aligned with human gameplay review under the Usable-rate criterion. Across 111 tasks, 12 coding agents, and 14 evaluation configurations, WebGameBench separates current systems: the best configuration reaches a 76.9% usable rate but only a 20.2% excellent rate. This gap shows that crossing the minimum playable-delivery threshold is still far from complete requirement satisfaction. To our knowledge, WebGameBench is the first requirement-to-application benchmark for browser-native game delivery that validates delivered-application runtime labels against independent human gameplay review under the Usable-rate criterion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_17637 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games Zhang, Wenyu You, Guoliang Tianlun Zhao, Haotian Zhu, Tianshu Wang, Haoran Tang, Xiaoxuan Dai, Mingyang Gu, Jingnan Dong, Daxiang Wu, Jianmin Artificial Intelligence cs.AI Coding agents are increasingly used as application builders, yet many evaluations still focus on source code, repository-level tests, or intermediate traces rather than the delivered application. We introduce WebGameBench, a requirement-to-application benchmark that evaluates whether coding agents can turn a frozen Structured WebGame Specification into a browser-accessible game. Browser-native games provide a compact but behavior-dense testbed: even simple games require coordinated input handling, spatial mapping, rule execution, state transitions, terminal conditions, restart behavior, and visible feedback. In WebGameBench, each generated artifact is built, served, and exposed as a browser-accessible application under a unified deployment protocol. A runtime evaluator then interacts with the delivered game in a real browser and assigns a three-way label: EXCELLENT, USABLE, or UNUSABLE. On a human-reviewed subset, the runtime label is broadly aligned with human gameplay review under the Usable-rate criterion. Across 111 tasks, 12 coding agents, and 14 evaluation configurations, WebGameBench separates current systems: the best configuration reaches a 76.9% usable rate but only a 20.2% excellent rate. This gap shows that crossing the minimum playable-delivery threshold is still far from complete requirement satisfaction. To our knowledge, WebGameBench is the first requirement-to-application benchmark for browser-native game delivery that validates delivered-application runtime labels against independent human gameplay review under the Usable-rate criterion. |
| title | WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games |
| topic | Artificial Intelligence cs.AI |
| url | https://arxiv.org/abs/2605.17637 |