S$^2$NN: Sub-bit Spiking Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917038450016256 |
|---|---|
| author | Wei, Wenjie Zhang, Malu Zhang, Jieyuan Belatreche, Ammar Wang, Shuai Shan, Yimeng Liu, Hanwen Cao, Honglin Wang, Guoqing Yang, Yang Li, Haizhou |
| author_facet | Wei, Wenjie Zhang, Malu Zhang, Jieyuan Belatreche, Ammar Wang, Shuai Shan, Yimeng Liu, Hanwen Cao, Honglin Wang, Guoqing Yang, Yang Li, Haizhou |
| contents | Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks. To further explore the compression and acceleration potential of SNNs, we propose Sub-bit Spiking Neural Networks (S$^2$NNs) that represent weights with less than one bit. Specifically, we first establish an S$^2$NN baseline by leveraging the clustering patterns of kernels in well-trained binary SNNs. This baseline is highly efficient but suffers from \textit{outlier-induced codeword selection bias} during training. To mitigate this issue, we propose an \textit{outlier-aware sub-bit weight quantization} (OS-Quant) method, which optimizes codeword selection by identifying and adaptively scaling outliers. Furthermore, we propose a \textit{membrane potential-based feature distillation} (MPFD) method, improving the performance of highly compressed S$^2$NN via more precise guidance from a teacher model. Extensive results on vision tasks reveal that S$^2$NN outperforms existing quantized SNNs in both performance and efficiency, making it promising for edge computing applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24266 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | S$^2$NN: Sub-bit Spiking Neural Networks Wei, Wenjie Zhang, Malu Zhang, Jieyuan Belatreche, Ammar Wang, Shuai Shan, Yimeng Liu, Hanwen Cao, Honglin Wang, Guoqing Yang, Yang Li, Haizhou Computer Vision and Pattern Recognition Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks. To further explore the compression and acceleration potential of SNNs, we propose Sub-bit Spiking Neural Networks (S$^2$NNs) that represent weights with less than one bit. Specifically, we first establish an S$^2$NN baseline by leveraging the clustering patterns of kernels in well-trained binary SNNs. This baseline is highly efficient but suffers from \textit{outlier-induced codeword selection bias} during training. To mitigate this issue, we propose an \textit{outlier-aware sub-bit weight quantization} (OS-Quant) method, which optimizes codeword selection by identifying and adaptively scaling outliers. Furthermore, we propose a \textit{membrane potential-based feature distillation} (MPFD) method, improving the performance of highly compressed S$^2$NN via more precise guidance from a teacher model. Extensive results on vision tasks reveal that S$^2$NN outperforms existing quantized SNNs in both performance and efficiency, making it promising for edge computing applications. |
| title | S$^2$NN: Sub-bit Spiking Neural Networks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.24266 |