Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914866497847296 |
|---|---|
| author | Chang, Kaiyan Chen, Zhirong Zhou, Yunhao Zhu, Wenlong wang, kun Xu, Haobo Li, Cangyuan Wang, Mengdi Liang, Shengwen Li, Huawei Han, Yinhe Wang, Ying |
| author_facet | Chang, Kaiyan Chen, Zhirong Zhou, Yunhao Zhu, Wenlong wang, kun Xu, Haobo Li, Cangyuan Wang, Mengdi Liang, Shengwen Li, Huawei Han, Yinhe Wang, Ying |
| contents | Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribute essential contextual information critical to design intent for hardware architectures possessing spatial complexity, potentially surpassing the efficacy of natural-language-only inputs. Expanding upon this premise, our paper introduces an open-source benchmark for multi-modal generative models tailored for Verilog synthesis from visual-linguistic inputs, addressing both singular and complex modules. Additionally, we introduce an open-source visual and natural language Verilog query language framework to facilitate efficient and user-friendly multi-modal queries. To evaluate the performance of the proposed multi-modal hardware generative AI in Verilog generation tasks, we compare it with a popular method that relies solely on natural language. Our results demonstrate a significant accuracy improvement in the multi-modal generated Verilog compared to queries based solely on natural language. We hope to reveal a new approach to hardware design in the large-hardware-design-model era, thereby fostering a more diversified and productive approach to hardware design. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08473 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation Chang, Kaiyan Chen, Zhirong Zhou, Yunhao Zhu, Wenlong wang, kun Xu, Haobo Li, Cangyuan Wang, Mengdi Liang, Shengwen Li, Huawei Han, Yinhe Wang, Ying Hardware Architecture Artificial Intelligence Natural language interfaces have exhibited considerable potential in the automation of Verilog generation derived from high-level specifications through the utilization of large language models, garnering significant attention. Nevertheless, this paper elucidates that visual representations contribute essential contextual information critical to design intent for hardware architectures possessing spatial complexity, potentially surpassing the efficacy of natural-language-only inputs. Expanding upon this premise, our paper introduces an open-source benchmark for multi-modal generative models tailored for Verilog synthesis from visual-linguistic inputs, addressing both singular and complex modules. Additionally, we introduce an open-source visual and natural language Verilog query language framework to facilitate efficient and user-friendly multi-modal queries. To evaluate the performance of the proposed multi-modal hardware generative AI in Verilog generation tasks, we compare it with a popular method that relies solely on natural language. Our results demonstrate a significant accuracy improvement in the multi-modal generated Verilog compared to queries based solely on natural language. We hope to reveal a new approach to hardware design in the large-hardware-design-model era, thereby fostering a more diversified and productive approach to hardware design. |
| title | Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation |
| topic | Hardware Architecture Artificial Intelligence |
| url | https://arxiv.org/abs/2407.08473 |