NL-DPE: An Analog In-memory Non-Linear Dot Product Engine for Efficient CNN and LLM Inference
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908660592017408 |
|---|---|
| author | Zhao, Lei Buonanno, Luca Gajjar, Archit Moon, John Natarajan, Aishwarya Serebryakov, Sergey Roth, Ron M. Sheng, Xia Zhang, Youtao Faraboschi, Paolo Ignowski, Jim Pedretti, Giacomo |
| author_facet | Zhao, Lei Buonanno, Luca Gajjar, Archit Moon, John Natarajan, Aishwarya Serebryakov, Sergey Roth, Ron M. Sheng, Xia Zhang, Youtao Faraboschi, Paolo Ignowski, Jim Pedretti, Giacomo |
| contents | Resistive Random Access Memory (RRAM) based in-memory computing (IMC) accelerators offer significant performance and energy advantages for deep neural networks (DNNs), but face three major limitations: (1) they support only \textit{static} dot-product operations and cannot accelerate arbitrary non-linear functions or data-dependent multiplications essential to modern LLMs; (2) they demand large, power-hungry analog-to-digital converter (ADC) circuits; and (3) mapping model weights to device conductance introduces errors from cell nonidealities. These challenges hinder scalable and accurate IMC acceleration as models grow.
We propose NL-DPE, a Non-Linear Dot Product Engine that overcomes these barriers. NL-DPE augments crosspoint arrays with RRAM-based Analog Content Addressable Memory (ACAM) to execute arbitrary non-linear functions and data-dependent matrix multiplications in the analog domain by transforming them into decision trees, fully eliminating ADCs. To address device noise, NL-DPE uses software-based Noise Aware Fine-tuning (NAF), requiring no in-device calibration. Experiments show that NL-DPE delivers 28X energy efficiency and 249X speedup over a GPU baseline, and 22X energy efficiency and 245X speedup over existing IMC accelerators, while maintaining high accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13950 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NL-DPE: An Analog In-memory Non-Linear Dot Product Engine for Efficient CNN and LLM Inference Zhao, Lei Buonanno, Luca Gajjar, Archit Moon, John Natarajan, Aishwarya Serebryakov, Sergey Roth, Ron M. Sheng, Xia Zhang, Youtao Faraboschi, Paolo Ignowski, Jim Pedretti, Giacomo Hardware Architecture Resistive Random Access Memory (RRAM) based in-memory computing (IMC) accelerators offer significant performance and energy advantages for deep neural networks (DNNs), but face three major limitations: (1) they support only \textit{static} dot-product operations and cannot accelerate arbitrary non-linear functions or data-dependent multiplications essential to modern LLMs; (2) they demand large, power-hungry analog-to-digital converter (ADC) circuits; and (3) mapping model weights to device conductance introduces errors from cell nonidealities. These challenges hinder scalable and accurate IMC acceleration as models grow. We propose NL-DPE, a Non-Linear Dot Product Engine that overcomes these barriers. NL-DPE augments crosspoint arrays with RRAM-based Analog Content Addressable Memory (ACAM) to execute arbitrary non-linear functions and data-dependent matrix multiplications in the analog domain by transforming them into decision trees, fully eliminating ADCs. To address device noise, NL-DPE uses software-based Noise Aware Fine-tuning (NAF), requiring no in-device calibration. Experiments show that NL-DPE delivers 28X energy efficiency and 249X speedup over a GPU baseline, and 22X energy efficiency and 245X speedup over existing IMC accelerators, while maintaining high accuracy. |
| title | NL-DPE: An Analog In-memory Non-Linear Dot Product Engine for Efficient CNN and LLM Inference |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2511.13950 |