Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.17582 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911768310185984 |
|---|---|
| author | Zhai, Yifeng Li, Bing Yan, Bonan Wang, Jing |
| author_facet | Zhai, Yifeng Li, Bing Yan, Bonan Wang, Jing |
| contents | RRAM crossbars have been studied to construct in-memory accelerators for neural network applications due to their in-situ computing capability. However, prior RRAM-based accelerators show efficiency degradation when executing the popular attention models. We observed that the frequent softmax operations arise as the efficiency bottleneck and also are insensitive to computing precision. Thus, we propose STAR, which boosts the computing efficiency with an efficient RRAM-based softmax engine and a fine-grained global pipeline for the attention models. Specifically, STAR exploits the versatility and flexibility of RRAM crossbars to trade off the model accuracy and hardware efficiency. The experimental results evaluated on several datasets show STAR achieves up to 30.63x and 1.31x computing efficiency improvements over the GPU and the state-of-the-art RRAM-based attention accelerators, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_17582 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | STAR: An Efficient Softmax Engine for Attention Model with RRAM Crossbar Zhai, Yifeng Li, Bing Yan, Bonan Wang, Jing Hardware Architecture RRAM crossbars have been studied to construct in-memory accelerators for neural network applications due to their in-situ computing capability. However, prior RRAM-based accelerators show efficiency degradation when executing the popular attention models. We observed that the frequent softmax operations arise as the efficiency bottleneck and also are insensitive to computing precision. Thus, we propose STAR, which boosts the computing efficiency with an efficient RRAM-based softmax engine and a fine-grained global pipeline for the attention models. Specifically, STAR exploits the versatility and flexibility of RRAM crossbars to trade off the model accuracy and hardware efficiency. The experimental results evaluated on several datasets show STAR achieves up to 30.63x and 1.31x computing efficiency improvements over the GPU and the state-of-the-art RRAM-based attention accelerators, respectively. |
| title | STAR: An Efficient Softmax Engine for Attention Model with RRAM Crossbar |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2401.17582 |