Continual Learning through Control Minimization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915773998432256 |
|---|---|
| author | de Haan, Sander Taoudi-Benchekroun, Yassine Aceituno, Pau Vilimelis Grewe, Benjamin F. |
| author_facet | de Haan, Sander Taoudi-Benchekroun, Yassine Aceituno, Pau Vilimelis Grewe, Benjamin F. |
| contents | Catastrophic forgetting remains a fundamental challenge for neural networks when tasks are trained sequentially. In this work, we reformulate continual learning as a control problem where learning and preservation signals compete within neural activity dynamics. We convert regularization penalties into preservation signals that protect prior-task representations. Learning then proceeds by minimizing the control effort required to integrate new tasks while competing with the preservation of prior tasks. At equilibrium, the neural activities produce weight updates that implicitly encode the full prior-task curvature, a property we term the continual-natural gradient, requiring no explicit curvature storage. Experiments confirm that our learning framework recovers true prior-task curvature and enables task discrimination, outperforming existing methods on standard benchmarks without replay. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_04542 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Continual Learning through Control Minimization de Haan, Sander Taoudi-Benchekroun, Yassine Aceituno, Pau Vilimelis Grewe, Benjamin F. Machine Learning Artificial Intelligence Catastrophic forgetting remains a fundamental challenge for neural networks when tasks are trained sequentially. In this work, we reformulate continual learning as a control problem where learning and preservation signals compete within neural activity dynamics. We convert regularization penalties into preservation signals that protect prior-task representations. Learning then proceeds by minimizing the control effort required to integrate new tasks while competing with the preservation of prior tasks. At equilibrium, the neural activities produce weight updates that implicitly encode the full prior-task curvature, a property we term the continual-natural gradient, requiring no explicit curvature storage. Experiments confirm that our learning framework recovers true prior-task curvature and enables task discrimination, outperforming existing methods on standard benchmarks without replay. |
| title | Continual Learning through Control Minimization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.04542 |