Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910915320872960 |
|---|---|
| author | Liu, Geng Zhu, Fei Feng, Rong Yi, Zhiqiang Wang, Shiqi Meng, Gaofeng Zhang, Zhaoxiang |
| author_facet | Liu, Geng Zhu, Fei Feng, Rong Yi, Zhiqiang Wang, Shiqi Meng, Gaofeng Zhang, Zhaoxiang |
| contents | Humans and most animals inherently possess a distinctive capacity to continually acquire novel experiences and accumulate worldly knowledge over time. This ability, termed continual learning, is also critical for deep neural networks (DNNs) to adapt to the dynamically evolving world in open environments. However, DNNs notoriously suffer from catastrophic forgetting of previously learned knowledge when trained on sequential tasks. In this work, inspired by the interactive human memory and learning system, we propose a novel biomimetic continual learning framework that integrates semi-parametric memory and the wake-sleep consolidation mechanism. For the first time, our method enables deep neural networks to retain high performance on novel tasks while maintaining prior knowledge in real-world challenging continual learning scenarios, e.g., class-incremental learning on ImageNet. This study demonstrates that emulating biological intelligence provides a promising path to enable deep neural networks with continual learning capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_14727 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning Liu, Geng Zhu, Fei Feng, Rong Yi, Zhiqiang Wang, Shiqi Meng, Gaofeng Zhang, Zhaoxiang Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Humans and most animals inherently possess a distinctive capacity to continually acquire novel experiences and accumulate worldly knowledge over time. This ability, termed continual learning, is also critical for deep neural networks (DNNs) to adapt to the dynamically evolving world in open environments. However, DNNs notoriously suffer from catastrophic forgetting of previously learned knowledge when trained on sequential tasks. In this work, inspired by the interactive human memory and learning system, we propose a novel biomimetic continual learning framework that integrates semi-parametric memory and the wake-sleep consolidation mechanism. For the first time, our method enables deep neural networks to retain high performance on novel tasks while maintaining prior knowledge in real-world challenging continual learning scenarios, e.g., class-incremental learning on ImageNet. This study demonstrates that emulating biological intelligence provides a promising path to enable deep neural networks with continual learning capabilities. |
| title | Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.14727 |