MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917875358367744 |
|---|---|
| author | Wan, Yuxuan Dong, Yi Xiao, Jingyu Huo, Yintong Wang, Wenxuan Lyu, Michael R. |
| author_facet | Wan, Yuxuan Dong, Yi Xiao, Jingyu Huo, Yintong Wang, Wenxuan Lyu, Michael R. |
| contents | Multi-page websites dominate modern web development. However, existing design-to-code methods rely on simplified assumptions, limiting to single-page, self-contained webpages without external resource connection. To address this gap, we introduce the Multi-Page Resource-Aware Webpage (MRWeb) generation task, which transforms UI designs into multi-page, functional web UIs with internal/external navigation, image loading, and backend routing. We propose a novel resource list data structure to track resources, links, and design components. Our study applies existing methods to the MRWeb problem using a newly curated dataset of 500 websites (300 synthetic, 200 real-world). Specifically, we identify the best metric to evaluate the similarity of the web UI, assess the impact of the resource list on MRWeb generation, analyze MLLM limitations, and evaluate the effectiveness of the MRWeb tool in real-world workflows. The results show that resource lists boost navigation functionality from 0% to 66%-80% while facilitating visual similarity. Our proposed metrics and evaluation framework provide new insights into MLLM performance on MRWeb tasks. We release the MRWeb tool, dataset, and evaluation framework to promote further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15310 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs Wan, Yuxuan Dong, Yi Xiao, Jingyu Huo, Yintong Wang, Wenxuan Lyu, Michael R. Software Engineering Artificial Intelligence Information Retrieval Multi-page websites dominate modern web development. However, existing design-to-code methods rely on simplified assumptions, limiting to single-page, self-contained webpages without external resource connection. To address this gap, we introduce the Multi-Page Resource-Aware Webpage (MRWeb) generation task, which transforms UI designs into multi-page, functional web UIs with internal/external navigation, image loading, and backend routing. We propose a novel resource list data structure to track resources, links, and design components. Our study applies existing methods to the MRWeb problem using a newly curated dataset of 500 websites (300 synthetic, 200 real-world). Specifically, we identify the best metric to evaluate the similarity of the web UI, assess the impact of the resource list on MRWeb generation, analyze MLLM limitations, and evaluate the effectiveness of the MRWeb tool in real-world workflows. The results show that resource lists boost navigation functionality from 0% to 66%-80% while facilitating visual similarity. Our proposed metrics and evaluation framework provide new insights into MLLM performance on MRWeb tasks. We release the MRWeb tool, dataset, and evaluation framework to promote further research. |
| title | MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs |
| topic | Software Engineering Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2412.15310 |