Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915072046006272 |
|---|---|
| author | Dequino, Alberto Carpegna, Alessio Nadalini, Davide Savino, Alessandro Benini, Luca Di Carlo, Stefano Conti, Francesco |
| author_facet | Dequino, Alberto Carpegna, Alessio Nadalini, Davide Savino, Alessandro Benini, Luca Di Carlo, Stefano Conti, Francesco |
| contents | Rehearsal-based Continual Learning (CL) has been intensely investigated in Deep Neural Networks (DNNs). However, its application in Spiking Neural Networks (SNNs) has not been explored in depth. In this paper we introduce the first memory-efficient implementation of Latent Replay (LR)-based CL for SNNs, designed to seamlessly integrate with resource-constrained devices. LRs combine new samples with latent representations of previously learned data, to mitigate forgetting. Experiments on the Heidelberg SHD dataset with Sample and Class-Incremental tasks reach a Top-1 accuracy of 92.5% and 92%, respectively, without forgetting the previously learned information. Furthermore, we minimize the LRs' requirements by applying a time-domain compression, reducing by two orders of magnitude their memory requirement, with respect to a naive rehearsal setup, with a maximum accuracy drop of 4%. On a Multi-Class-Incremental task, our SNN learns 10 new classes from an initial set of 10, reaching a Top-1 accuracy of 78.4% on the full test set. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03111 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks Dequino, Alberto Carpegna, Alessio Nadalini, Davide Savino, Alessandro Benini, Luca Di Carlo, Stefano Conti, Francesco Neural and Evolutionary Computing Artificial Intelligence Emerging Technologies Machine Learning Rehearsal-based Continual Learning (CL) has been intensely investigated in Deep Neural Networks (DNNs). However, its application in Spiking Neural Networks (SNNs) has not been explored in depth. In this paper we introduce the first memory-efficient implementation of Latent Replay (LR)-based CL for SNNs, designed to seamlessly integrate with resource-constrained devices. LRs combine new samples with latent representations of previously learned data, to mitigate forgetting. Experiments on the Heidelberg SHD dataset with Sample and Class-Incremental tasks reach a Top-1 accuracy of 92.5% and 92%, respectively, without forgetting the previously learned information. Furthermore, we minimize the LRs' requirements by applying a time-domain compression, reducing by two orders of magnitude their memory requirement, with respect to a naive rehearsal setup, with a maximum accuracy drop of 4%. On a Multi-Class-Incremental task, our SNN learns 10 new classes from an initial set of 10, reaching a Top-1 accuracy of 78.4% on the full test set. |
| title | Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks |
| topic | Neural and Evolutionary Computing Artificial Intelligence Emerging Technologies Machine Learning |
| url | https://arxiv.org/abs/2407.03111 |