DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909595232894976 |
|---|---|
| author | Yuan, Mingyue Chen, Jieshan Xing, Zhenchang Quigley, Aaron Luo, Yuyu Luo, Tianqi Mohammadi, Gelareh Lu, Qinghua Zhu, Liming |
| author_facet | Yuan, Mingyue Chen, Jieshan Xing, Zhenchang Quigley, Aaron Luo, Yuyu Luo, Tianqi Mohammadi, Gelareh Lu, Qinghua Zhu, Liming |
| contents | The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's V0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_01606 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models Yuan, Mingyue Chen, Jieshan Xing, Zhenchang Quigley, Aaron Luo, Yuyu Luo, Tianqi Mohammadi, Gelareh Lu, Qinghua Zhu, Liming Software Engineering D.2.2 The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's V0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics. |
| title | DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models |
| topic | Software Engineering D.2.2 |
| url | https://arxiv.org/abs/2411.01606 |