Implementation of transformer-based LLMs with large-scale optoelectronic neurons on a CMOS compatible platform
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915879660290048 |
|---|---|
| author | Na, Neil Cheng, Chih-Hao Hsu, Shou-Chen Liang, Che-Fu Lin, Chung-Chih Na, Nathaniel Y. Shieh, Andrew I. Chen, Erik Rong, Haisheng Soref, Richard A. |
| author_facet | Na, Neil Cheng, Chih-Hao Hsu, Shou-Chen Liang, Che-Fu Lin, Chung-Chih Na, Nathaniel Y. Shieh, Andrew I. Chen, Erik Rong, Haisheng Soref, Richard A. |
| contents | The recent rapid deployment of datacenter infrastructures for performing large language models (LLMs) and related artificial intelligence (AI) applications in the clouds is predicted to incur an exponentially growing energy consumption in the near-term future. In this paper, we propose and analyze the implementation of the transformer model, which is the cornerstone of the modern LLMs, with novel large-scale optoelectronic neurons (OENs) constructed over a complementary metal-oxide-semiconductor (CMOS) compatible platform. With all of the required optoelectronic devices and electronic circuits integrated in a chiplet only about 2 cm by 3 cm in size, 175 billon parameters in the case of GPT-3 are shown to perform inference at an unprecedented speed of 12.6 POPS using only 40 nm CMOS process node, orchestrated by an optoelectronic version of systolic array with no data skew and negligible propagation delay, along with a high power efficiency of 74 TOPS/W and a high area efficiency of 19 TOPS/mm2. The influence of the quantization formats and the hardware induced errors are numerically investigated, and are shown to have a minimal impact. Our study presents a new yet practical path toward analog neural processing units (NPUs) to complement existing digital processing units. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04136 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Implementation of transformer-based LLMs with large-scale optoelectronic neurons on a CMOS compatible platform Na, Neil Cheng, Chih-Hao Hsu, Shou-Chen Liang, Che-Fu Lin, Chung-Chih Na, Nathaniel Y. Shieh, Andrew I. Chen, Erik Rong, Haisheng Soref, Richard A. Emerging Technologies Applied Physics Optics The recent rapid deployment of datacenter infrastructures for performing large language models (LLMs) and related artificial intelligence (AI) applications in the clouds is predicted to incur an exponentially growing energy consumption in the near-term future. In this paper, we propose and analyze the implementation of the transformer model, which is the cornerstone of the modern LLMs, with novel large-scale optoelectronic neurons (OENs) constructed over a complementary metal-oxide-semiconductor (CMOS) compatible platform. With all of the required optoelectronic devices and electronic circuits integrated in a chiplet only about 2 cm by 3 cm in size, 175 billon parameters in the case of GPT-3 are shown to perform inference at an unprecedented speed of 12.6 POPS using only 40 nm CMOS process node, orchestrated by an optoelectronic version of systolic array with no data skew and negligible propagation delay, along with a high power efficiency of 74 TOPS/W and a high area efficiency of 19 TOPS/mm2. The influence of the quantization formats and the hardware induced errors are numerically investigated, and are shown to have a minimal impact. Our study presents a new yet practical path toward analog neural processing units (NPUs) to complement existing digital processing units. |
| title | Implementation of transformer-based LLMs with large-scale optoelectronic neurons on a CMOS compatible platform |
| topic | Emerging Technologies Applied Physics Optics |
| url | https://arxiv.org/abs/2511.04136 |