Can We Understand Plasticity Through Neural Collapse?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911826383470592 |
|---|---|
| author | Bonifazi, Guglielmo Chalas, Iason Hess, Gian Łucki, Jakub |
| author_facet | Bonifazi, Guglielmo Chalas, Iason Hess, Gian Łucki, Jakub |
| contents | This paper explores the connection between two recently identified phenomena in deep learning: plasticity loss and neural collapse. We analyze their correlation in different scenarios, revealing a significant association during the initial training phase on the first task. Additionally, we introduce a regularization approach to mitigate neural collapse, demonstrating its effectiveness in alleviating plasticity loss in this specific setting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02719 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Can We Understand Plasticity Through Neural Collapse? Bonifazi, Guglielmo Chalas, Iason Hess, Gian Łucki, Jakub Machine Learning Artificial Intelligence This paper explores the connection between two recently identified phenomena in deep learning: plasticity loss and neural collapse. We analyze their correlation in different scenarios, revealing a significant association during the initial training phase on the first task. Additionally, we introduce a regularization approach to mitigate neural collapse, demonstrating its effectiveness in alleviating plasticity loss in this specific setting. |
| title | Can We Understand Plasticity Through Neural Collapse? |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2404.02719 |