PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916672492797952 |
|---|---|
| author | Malekar, Jinendra Chandarana, Peyton Amin, Md Hasibul Elbtity, Mohammed E. Zand, Ramtin |
| author_facet | Malekar, Jinendra Chandarana, Peyton Amin, Md Hasibul Elbtity, Mohammed E. Zand, Ramtin |
| contents | In this paper, we propose PIM-LLM, a hybrid architecture developed to accelerate 1-bit large language models (LLMs). PIM-LLM leverages analog processing-in-memory (PIM) architectures and digital systolic arrays to accelerate low-precision matrix multiplication (MatMul) operations in projection layers and high-precision MatMul operations in attention heads of 1-bit LLMs, respectively. Our design achieves up to roughly 80x improvement in tokens per second and a 70% increase in tokens per joule compared to conventional hardware accelerators. Additionally, PIM-LLM outperforms previous PIM-based LLM accelerators, setting a new benchmark with at least 2x and 5x improvement in GOPS and GOPS/W, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_01994 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs Malekar, Jinendra Chandarana, Peyton Amin, Md Hasibul Elbtity, Mohammed E. Zand, Ramtin Hardware Architecture Artificial Intelligence In this paper, we propose PIM-LLM, a hybrid architecture developed to accelerate 1-bit large language models (LLMs). PIM-LLM leverages analog processing-in-memory (PIM) architectures and digital systolic arrays to accelerate low-precision matrix multiplication (MatMul) operations in projection layers and high-precision MatMul operations in attention heads of 1-bit LLMs, respectively. Our design achieves up to roughly 80x improvement in tokens per second and a 70% increase in tokens per joule compared to conventional hardware accelerators. Additionally, PIM-LLM outperforms previous PIM-based LLM accelerators, setting a new benchmark with at least 2x and 5x improvement in GOPS and GOPS/W, respectively. |
| title | PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs |
| topic | Hardware Architecture Artificial Intelligence |
| url | https://arxiv.org/abs/2504.01994 |