FireFly-S: Exploiting Dual-Side Sparsity for Spiking Neural Networks Acceleration with Reconfigurable Spatial Architecture
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912856108171264 |
|---|---|
| author | Li, Tenglong Li, Jindong Shen, Guobin Zhao, Dongcheng Zhang, Qian Zeng, Yi |
| author_facet | Li, Tenglong Li, Jindong Shen, Guobin Zhao, Dongcheng Zhang, Qian Zeng, Yi |
| contents | Spiking Neural Networks (SNNs), with brain-inspired structure using discrete spikes instead of continuous activations, are gaining attention for their efficient processing on neuromorphic chips. While current SNN hardware accelerators often prioritize temporal spike sparsity, exploiting sparse synaptic weights offers significant untapped potential for even greater efficiency. To address this, we propose FireFly-S, a Sparse extension of the FireFly series. This co-optimized software-hardware design focuses on leveraging dual-side sparsity for acceleration. On the software side, we propose a algorithmic optimization framework that combines gradient rewiring for pruning and modified Learned Step Size Quantization (LSQ) for SNNs, achieving a weight sparsity exceeding 85\% and enabling efficient 4-bit quantization with negligible accuracy loss. On the hardware side, we present an efficient dual-side sparsity detector employing a Bitmap-based sparse decoding logic to pinpoint the positions of non-zero weights and input spikes. The logic allows for direct bypassing of redundant computations, thereby enhancing computational efficiency. Different from the overlay architecture adopted by previous FireFly series, we adopt a parametric spatial architecture with inter-layer pipelining that can fully exploit the fine-grained programmability and reconfigurability of Field-Programmable Gate Arrays (FPGAs), enabling fast deployment for various models. A spatial-temporal dataflow is also proposed to support such inter-layer pipelining and avoid long-term temporal dependencies. In experiments conducted on the MNIST, DVS-Gesture and CIFAR-10 datasets, the FireFly-S model achieves 85--95\% sparsity with 4-bit quantization and the hardware accelerator effectively leverages the dual-side sparsity, delivering performance metrics of 10,047~FPS/W on MNIST, 3,683~FPS/W on DVS-Gesture, and 2,327~FPS/W on CIFAR-10. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_15578 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FireFly-S: Exploiting Dual-Side Sparsity for Spiking Neural Networks Acceleration with Reconfigurable Spatial Architecture Li, Tenglong Li, Jindong Shen, Guobin Zhao, Dongcheng Zhang, Qian Zeng, Yi Hardware Architecture Spiking Neural Networks (SNNs), with brain-inspired structure using discrete spikes instead of continuous activations, are gaining attention for their efficient processing on neuromorphic chips. While current SNN hardware accelerators often prioritize temporal spike sparsity, exploiting sparse synaptic weights offers significant untapped potential for even greater efficiency. To address this, we propose FireFly-S, a Sparse extension of the FireFly series. This co-optimized software-hardware design focuses on leveraging dual-side sparsity for acceleration. On the software side, we propose a algorithmic optimization framework that combines gradient rewiring for pruning and modified Learned Step Size Quantization (LSQ) for SNNs, achieving a weight sparsity exceeding 85\% and enabling efficient 4-bit quantization with negligible accuracy loss. On the hardware side, we present an efficient dual-side sparsity detector employing a Bitmap-based sparse decoding logic to pinpoint the positions of non-zero weights and input spikes. The logic allows for direct bypassing of redundant computations, thereby enhancing computational efficiency. Different from the overlay architecture adopted by previous FireFly series, we adopt a parametric spatial architecture with inter-layer pipelining that can fully exploit the fine-grained programmability and reconfigurability of Field-Programmable Gate Arrays (FPGAs), enabling fast deployment for various models. A spatial-temporal dataflow is also proposed to support such inter-layer pipelining and avoid long-term temporal dependencies. In experiments conducted on the MNIST, DVS-Gesture and CIFAR-10 datasets, the FireFly-S model achieves 85--95\% sparsity with 4-bit quantization and the hardware accelerator effectively leverages the dual-side sparsity, delivering performance metrics of 10,047~FPS/W on MNIST, 3,683~FPS/W on DVS-Gesture, and 2,327~FPS/W on CIFAR-10. |
| title | FireFly-S: Exploiting Dual-Side Sparsity for Spiking Neural Networks Acceleration with Reconfigurable Spatial Architecture |
| topic | Hardware Architecture |
| url | https://arxiv.org/abs/2408.15578 |