Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913726542643200 |
|---|---|
| author | Che, Kaiwei Fang, Wei Ma, Zhengyu Huang, Yifan Xue, Peng Yuan, Li Masquelier, Timothée Tian, Yonghong |
| author_facet | Che, Kaiwei Fang, Wei Ma, Zhengyu Huang, Yifan Xue, Peng Yuan, Li Masquelier, Timothée Tian, Yonghong |
| contents | Spiking Neural Networks (SNNs), with their event-driven and biologically inspired operation, are well-suited for energy-efficient neuromorphic hardware. Neural coding, critical to SNNs, determines how information is represented via spikes. Time-to-First-Spike (TTFS) coding, which uses a single spike per neuron, offers extreme sparsity and energy efficiency but suffers from unstable training and low accuracy due to its sparse firing. To address these challenges, we propose a training framework incorporating parameter initialization, training normalization, temporal output decoding, and pooling layer re-evaluation. The proposed parameter initialization and training normalization mitigate signal diminishing and gradient vanishing to stabilize training. The output decoding method aggregates temporal spikes to encourage earlier firing, thereby reducing the latency. The re-evaluation of the pooling layer indicates that average-pooling keeps the single-spike characteristic and that max-pooling should be avoided. Experiments show the framework stabilizes and accelerates training, reduces latency, and achieves state-of-the-art accuracy for TTFS SNNs on MNIST (99.48%), Fashion-MNIST (92.90%), CIFAR10 (90.56%), and DVS Gesture (95.83%). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_23619 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch Che, Kaiwei Fang, Wei Ma, Zhengyu Huang, Yifan Xue, Peng Yuan, Li Masquelier, Timothée Tian, Yonghong Neural and Evolutionary Computing Spiking Neural Networks (SNNs), with their event-driven and biologically inspired operation, are well-suited for energy-efficient neuromorphic hardware. Neural coding, critical to SNNs, determines how information is represented via spikes. Time-to-First-Spike (TTFS) coding, which uses a single spike per neuron, offers extreme sparsity and energy efficiency but suffers from unstable training and low accuracy due to its sparse firing. To address these challenges, we propose a training framework incorporating parameter initialization, training normalization, temporal output decoding, and pooling layer re-evaluation. The proposed parameter initialization and training normalization mitigate signal diminishing and gradient vanishing to stabilize training. The output decoding method aggregates temporal spikes to encourage earlier firing, thereby reducing the latency. The re-evaluation of the pooling layer indicates that average-pooling keeps the single-spike characteristic and that max-pooling should be avoided. Experiments show the framework stabilizes and accelerates training, reduces latency, and achieves state-of-the-art accuracy for TTFS SNNs on MNIST (99.48%), Fashion-MNIST (92.90%), CIFAR10 (90.56%), and DVS Gesture (95.83%). |
| title | Efficiently Training Time-to-First-Spike Spiking Neural Networks from Scratch |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2410.23619 |