Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866907787563368448 |
|---|---|
| author | Fu, Weimin Li, Shijie Zhao, Yifang Ma, Haocheng Dutta, Raj Zhang, Xuan Yang, Kaichen Jin, Yier Guo, Xiaolong |
| author_facet | Fu, Weimin Li, Shijie Zhao, Yifang Ma, Haocheng Dutta, Raj Zhang, Xuan Yang, Kaichen Jin, Yier Guo, Xiaolong |
| contents | In the rapidly evolving semiconductor industry, where research, design, verification, and manufacturing are intricately linked, the potential of Large Language Models to revolutionize hardware design and security verification is immense. The primary challenge, however, lies in the complexity of hardware specific issues that are not adequately addressed by the natural language or software code knowledge typically acquired during the pretraining stage. Additionally, the scarcity of datasets specific to the hardware domain poses a significant hurdle in developing a foundational model. Addressing these challenges, this paper introduces Hardware Phi 1.5B, an innovative large language model specifically tailored for the hardware domain of the semiconductor industry. We have developed a specialized, tiered dataset comprising small, medium, and large subsets and focused our efforts on pretraining using the medium dataset. This approach harnesses the compact yet efficient architecture of the Phi 1.5B model. The creation of this first pretrained, hardware domain specific large language model marks a significant advancement, offering improved performance in hardware design and verification tasks and illustrating a promising path forward for AI applications in the semiconductor sector. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01728 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge Fu, Weimin Li, Shijie Zhao, Yifang Ma, Haocheng Dutta, Raj Zhang, Xuan Yang, Kaichen Jin, Yier Guo, Xiaolong Computation and Language Artificial Intelligence Hardware Architecture In the rapidly evolving semiconductor industry, where research, design, verification, and manufacturing are intricately linked, the potential of Large Language Models to revolutionize hardware design and security verification is immense. The primary challenge, however, lies in the complexity of hardware specific issues that are not adequately addressed by the natural language or software code knowledge typically acquired during the pretraining stage. Additionally, the scarcity of datasets specific to the hardware domain poses a significant hurdle in developing a foundational model. Addressing these challenges, this paper introduces Hardware Phi 1.5B, an innovative large language model specifically tailored for the hardware domain of the semiconductor industry. We have developed a specialized, tiered dataset comprising small, medium, and large subsets and focused our efforts on pretraining using the medium dataset. This approach harnesses the compact yet efficient architecture of the Phi 1.5B model. The creation of this first pretrained, hardware domain specific large language model marks a significant advancement, offering improved performance in hardware design and verification tasks and illustrating a promising path forward for AI applications in the semiconductor sector. |
| title | Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge |
| topic | Computation and Language Artificial Intelligence Hardware Architecture |
| url | https://arxiv.org/abs/2402.01728 |