Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.24709 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912718332624896 |
|---|---|
| author | Chen, Yang Liu, Minghao Shen, Yufan Li, Yunwen Huang, Tianyuan Fang, Xinyu Zheng, Tianyu Huang, Wenxuan Yang, Cheng Fu, Daocheng Mei, Jianbiao Wu, Rong Zhao, Yunfei Wen, Licheng Yang, Xuemeng Mao, Song Lin, Qunshu Yu, Zhi Shen, Yongliang Qiao, Yu Shi, Botian |
| author_facet | Chen, Yang Liu, Minghao Shen, Yufan Li, Yunwen Huang, Tianyuan Fang, Xinyu Zheng, Tianyu Huang, Wenxuan Yang, Cheng Fu, Daocheng Mei, Jianbiao Wu, Rong Zhao, Yunfei Wen, Licheng Yang, Xuemeng Mao, Song Lin, Qunshu Yu, Zhi Shen, Yongliang Qiao, Yu Shi, Botian |
| contents | The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic interactions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video. IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with standard web development practices, each task includes not only user interaction videos but also all crawled static assets (e.g., images, videos). This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional correctness and visual fidelity of generated webpages. Extensive experiments on 28 LVLMs reveal a significant challenge: the best model achieves an overall score of only 36.35%, as functional correctness (24.39% IFS) lags significantly behind visual fidelity (64.25% VFS). These results highlight critical limitations in current models' ability to reason about temporal dynamics and synthesize event-driven logic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available at https://github.com/SIGMME/IWR-Bench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24709 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video? Chen, Yang Liu, Minghao Shen, Yufan Li, Yunwen Huang, Tianyuan Fang, Xinyu Zheng, Tianyu Huang, Wenxuan Yang, Cheng Fu, Daocheng Mei, Jianbiao Wu, Rong Zhao, Yunfei Wen, Licheng Yang, Xuemeng Mao, Song Lin, Qunshu Yu, Zhi Shen, Yongliang Qiao, Yu Shi, Botian Computer Vision and Pattern Recognition The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic interactions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video. IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with standard web development practices, each task includes not only user interaction videos but also all crawled static assets (e.g., images, videos). This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional correctness and visual fidelity of generated webpages. Extensive experiments on 28 LVLMs reveal a significant challenge: the best model achieves an overall score of only 36.35%, as functional correctness (24.39% IFS) lags significantly behind visual fidelity (64.25% VFS). These results highlight critical limitations in current models' ability to reason about temporal dynamics and synthesize event-driven logic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available at https://github.com/SIGMME/IWR-Bench. |
| title | IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video? |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.24709 |