SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916491853561856 |
|---|---|
| author | Nguyen, Christopher Nguyen, William Suzuki, Atsushi Oku, Daisuke Phan, Hong An Dinh, Sang Nguyen, Zooey Ha, Anh Raghavan, Shruti Vo, Huy Nguyen, Thang Nguyen, Lan Hirayama, Yoshikuni |
| author_facet | Nguyen, Christopher Nguyen, William Suzuki, Atsushi Oku, Daisuke Phan, Hong An Dinh, Sang Nguyen, Zooey Ha, Anh Raghavan, Shruti Vo, Huy Nguyen, Thang Nguyen, Lan Hirayama, Yoshikuni |
| contents | Large Language Models (LLMs) have demonstrated the potential to address some issues within the semiconductor industry. However, they are often general-purpose models that lack the specialized knowledge needed to tackle the unique challenges of this sector, such as the intricate physics and chemistry of semiconductor devices and processes. SemiKong, the first industry-specific LLM for the semiconductor domain, provides a foundation that can be used to develop tailored proprietary models. With SemiKong 1.0, we aim to develop a foundational model capable of understanding etching problems at an expert level. Our key contributions include (a) curating a comprehensive corpus of semiconductor-related texts, (b) creating a foundational model with in-depth semiconductor knowledge, and (c) introducing a framework for integrating expert knowledge, thereby advancing the evaluation process of domain-specific AI models. Through fine-tuning a pre-trained LLM using our curated dataset, we have shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks. Our extensive experiments underscore the importance of developing domain-specific LLMs as a foundation for company- or tool-specific proprietary models, paving the way for further research and applications in the semiconductor domain. Code and dataset will be available at https://github.com/aitomatic/semikong |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_13802 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model Nguyen, Christopher Nguyen, William Suzuki, Atsushi Oku, Daisuke Phan, Hong An Dinh, Sang Nguyen, Zooey Ha, Anh Raghavan, Shruti Vo, Huy Nguyen, Thang Nguyen, Lan Hirayama, Yoshikuni Computation and Language Large Language Models (LLMs) have demonstrated the potential to address some issues within the semiconductor industry. However, they are often general-purpose models that lack the specialized knowledge needed to tackle the unique challenges of this sector, such as the intricate physics and chemistry of semiconductor devices and processes. SemiKong, the first industry-specific LLM for the semiconductor domain, provides a foundation that can be used to develop tailored proprietary models. With SemiKong 1.0, we aim to develop a foundational model capable of understanding etching problems at an expert level. Our key contributions include (a) curating a comprehensive corpus of semiconductor-related texts, (b) creating a foundational model with in-depth semiconductor knowledge, and (c) introducing a framework for integrating expert knowledge, thereby advancing the evaluation process of domain-specific AI models. Through fine-tuning a pre-trained LLM using our curated dataset, we have shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks. Our extensive experiments underscore the importance of developing domain-specific LLMs as a foundation for company- or tool-specific proprietary models, paving the way for further research and applications in the semiconductor domain. Code and dataset will be available at https://github.com/aitomatic/semikong |
| title | SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.13802 |