UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913854426972160 |
|---|---|
| author | Gui, Yi Li, Zhen Zhang, Zhongyi Wan, Yao Chen, Dongping Zhang, Hongyu Su, Yi Chen, Bohua Zhou, Xing Jiang, Wenbin Zhang, Xiangliang |
| author_facet | Gui, Yi Li, Zhen Zhang, Zhongyi Wan, Yao Chen, Dongping Zhang, Hongyu Su, Yi Chen, Bohua Zhou, Xing Jiang, Wenbin Zhang, Xiangliang |
| contents | Automating the synthesis of User Interfaces (UIs) plays a crucial role in enhancing productivity and accelerating the development lifecycle, reducing both development time and manual effort. Recently, the rapid development of Multimodal Large Language Models (MLLMs) has made it possible to generate front-end Hypertext Markup Language (HTML) code directly from webpage designs. However, real-world webpages encompass not only a diverse array of HTML tags but also complex stylesheets, resulting in significantly lengthy code. The lengthy code poses challenges for the performance and efficiency of MLLMs, especially in capturing the structural information of UI designs. To address these challenges, this paper proposes UICopilot, a novel approach to automating UI synthesis via hierarchical code generation from webpage designs. The core idea of UICopilot is to decompose the generation process into two stages: first, generating the coarse-grained HTML hierarchical structure, followed by the generation of fine-grained code. To validate the effectiveness of UICopilot, we conduct experiments on a real-world dataset, i.e., WebCode2M. Experimental results demonstrate that UICopilot significantly outperforms existing baselines in both automatic evaluation metrics and human evaluations. Specifically, statistical analysis reveals that the majority of human annotators prefer the webpages generated by UICopilot over those produced by GPT-4V. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_09904 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs Gui, Yi Li, Zhen Zhang, Zhongyi Wan, Yao Chen, Dongping Zhang, Hongyu Su, Yi Chen, Bohua Zhou, Xing Jiang, Wenbin Zhang, Xiangliang Software Engineering Automating the synthesis of User Interfaces (UIs) plays a crucial role in enhancing productivity and accelerating the development lifecycle, reducing both development time and manual effort. Recently, the rapid development of Multimodal Large Language Models (MLLMs) has made it possible to generate front-end Hypertext Markup Language (HTML) code directly from webpage designs. However, real-world webpages encompass not only a diverse array of HTML tags but also complex stylesheets, resulting in significantly lengthy code. The lengthy code poses challenges for the performance and efficiency of MLLMs, especially in capturing the structural information of UI designs. To address these challenges, this paper proposes UICopilot, a novel approach to automating UI synthesis via hierarchical code generation from webpage designs. The core idea of UICopilot is to decompose the generation process into two stages: first, generating the coarse-grained HTML hierarchical structure, followed by the generation of fine-grained code. To validate the effectiveness of UICopilot, we conduct experiments on a real-world dataset, i.e., WebCode2M. Experimental results demonstrate that UICopilot significantly outperforms existing baselines in both automatic evaluation metrics and human evaluations. Specifically, statistical analysis reveals that the majority of human annotators prefer the webpages generated by UICopilot over those produced by GPT-4V. |
| title | UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2505.09904 |