Learning without Isolation: Pathway Protection for Continual Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918033691246592 |
|---|---|
| author | Chen, Zhikang Wuerkaixi, Abudukelimu Cui, Sen Li, Haoxuan Li, Ding Zhang, Jingfeng Han, Bo Niu, Gang Liu, Houfang Yang, Yi Yang, Sifan Zhang, Changshui Ren, Tianling |
| author_facet | Chen, Zhikang Wuerkaixi, Abudukelimu Cui, Sen Li, Haoxuan Li, Ding Zhang, Jingfeng Han, Bo Niu, Gang Liu, Houfang Yang, Yi Yang, Sifan Zhang, Changshui Ren, Tianling |
| contents | Deep networks are prone to catastrophic forgetting during sequential task learning, i.e., losing the knowledge about old tasks upon learning new tasks. To this end, continual learning(CL) has emerged, whose existing methods focus mostly on regulating or protecting the parameters associated with the previous tasks. However, parameter protection is often impractical, since the size of parameters for storing the old-task knowledge increases linearly with the number of tasks, otherwise it is hard to preserve the parameters related to the old-task knowledge. In this work, we bring a dual opinion from neuroscience and physics to CL: in the whole networks, the pathways matter more than the parameters when concerning the knowledge acquired from the old tasks. Following this opinion, we propose a novel CL framework, learning without isolation(LwI), where model fusion is formulated as graph matching and the pathways occupied by the old tasks are protected without being isolated. Thanks to the sparsity of activation channels in a deep network, LwI can adaptively allocate available pathways for a new task, realizing pathway protection and addressing catastrophic forgetting in a parameter-efficient manner. Experiments on popular benchmark datasets demonstrate the superiority of the proposed LwI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18568 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning without Isolation: Pathway Protection for Continual Learning Chen, Zhikang Wuerkaixi, Abudukelimu Cui, Sen Li, Haoxuan Li, Ding Zhang, Jingfeng Han, Bo Niu, Gang Liu, Houfang Yang, Yi Yang, Sifan Zhang, Changshui Ren, Tianling Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Deep networks are prone to catastrophic forgetting during sequential task learning, i.e., losing the knowledge about old tasks upon learning new tasks. To this end, continual learning(CL) has emerged, whose existing methods focus mostly on regulating or protecting the parameters associated with the previous tasks. However, parameter protection is often impractical, since the size of parameters for storing the old-task knowledge increases linearly with the number of tasks, otherwise it is hard to preserve the parameters related to the old-task knowledge. In this work, we bring a dual opinion from neuroscience and physics to CL: in the whole networks, the pathways matter more than the parameters when concerning the knowledge acquired from the old tasks. Following this opinion, we propose a novel CL framework, learning without isolation(LwI), where model fusion is formulated as graph matching and the pathways occupied by the old tasks are protected without being isolated. Thanks to the sparsity of activation channels in a deep network, LwI can adaptively allocate available pathways for a new task, realizing pathway protection and addressing catastrophic forgetting in a parameter-efficient manner. Experiments on popular benchmark datasets demonstrate the superiority of the proposed LwI. |
| title | Learning without Isolation: Pathway Protection for Continual Learning |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.18568 |