PRIMAL: Processing-In-Memory Based Low-Rank Adaptation for LLM Inference Accelerator
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917211324547072 |
|---|---|
| author | Chong, Yue Jiet Wang, Yimin Wu, Zhen Fong, Xuanyao |
| author_facet | Chong, Yue Jiet Wang, Yimin Wu, Zhen Fong, Xuanyao |
| contents | This paper presents PRIMAL, a processing-in-memory (PIM) based large language model (LLM) inference accelerator with low-rank adaptation (LoRA). PRIMAL integrates heterogeneous PIM processing elements (PEs), interconnected by 2D-mesh inter-PE computational network (IPCN). A novel SRAM reprogramming and power gating (SRPG) scheme enables pipelined LoRA updates and sub-linear power scaling by overlapping reconfiguration with computation and gating idle resources. PRIMAL employs optimized spatial mapping and dataflow orchestration to minimize communication overhead, and achieves $1.5\times$ throughput and $25\times$ energy efficiency over NVIDIA H100 with LoRA rank 8 (Q,V) on Llama-13B. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_13628 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PRIMAL: Processing-In-Memory Based Low-Rank Adaptation for LLM Inference Accelerator Chong, Yue Jiet Wang, Yimin Wu, Zhen Fong, Xuanyao Hardware Architecture This paper presents PRIMAL, a processing-in-memory (PIM) based large language model (LLM) inference accelerator with low-rank adaptation (LoRA). PRIMAL integrates heterogeneous PIM processing elements (PEs), interconnected by 2D-mesh inter-PE computational network (IPCN). A novel SRAM reprogramming and power gating (SRPG) scheme enables pipelined LoRA updates and sub-linear power scaling by overlapping reconfiguration with computation and gating idle resources. PRIMAL employs optimized spatial mapping and dataflow orchestration to minimize communication overhead, and achieves $1.5\times$ throughput and $25\times$ energy efficiency over NVIDIA H100 with LoRA rank 8 (Q,V) on Llama-13B. |
| title | PRIMAL: Processing-In-Memory Based Low-Rank Adaptation for LLM Inference Accelerator |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2601.13628 |