31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914549032026112 |
|---|---|
| author | Dong, Pingcheng Tan, Yonghao Liu, Xuejiao Luo, Peng Liu, Yu Pang, Di Ma, Songchen Huang, Xijie Liu, Shih-Yang Zhang, Dong Lu, Zhichao Liang, Luhong Tsui, Chi-Ying Tu, Fengbin Zhao, Liang Cheng, Kwang-Ting |
| author_facet | Dong, Pingcheng Tan, Yonghao Liu, Xuejiao Luo, Peng Liu, Yu Pang, Di Ma, Songchen Huang, Xijie Liu, Shih-Yang Zhang, Dong Lu, Zhichao Liang, Luhong Tsui, Chi-Ying Tu, Fengbin Zhao, Liang Cheng, Kwang-Ting |
| contents | This work presents a 55nm speculative decoding-based LLM accelerator with bumping-based face-to-face ReRAM-on-logic stacking technology. It features a local rotation unit for outlier-free low-bit quantization, a stacking-aware PNM architecture co-designed with blockwise vector quantization to reduce weight EMA overheads, and an adaptive parallel speculative decoding scheme with an out-of-order scheduler for high resource and bandwidth utilization. Our chip achieves 14.08-to-135.69token/s and 4.46-to-7.17x speedup over vanilla speculative decoding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_09375 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | 31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding Dong, Pingcheng Tan, Yonghao Liu, Xuejiao Luo, Peng Liu, Yu Pang, Di Ma, Songchen Huang, Xijie Liu, Shih-Yang Zhang, Dong Lu, Zhichao Liang, Luhong Tsui, Chi-Ying Tu, Fengbin Zhao, Liang Cheng, Kwang-Ting Hardware Architecture This work presents a 55nm speculative decoding-based LLM accelerator with bumping-based face-to-face ReRAM-on-logic stacking technology. It features a local rotation unit for outlier-free low-bit quantization, a stacking-aware PNM architecture co-designed with blockwise vector quantization to reduce weight EMA overheads, and an adaptive parallel speculative decoding scheme with an out-of-order scheduler for high resource and bandwidth utilization. Our chip achieves 14.08-to-135.69token/s and 4.46-to-7.17x speedup over vanilla speculative decoding. |
| title | 31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2605.09375 |