Learning Equi-angular Representations for Online Continual Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917628867510272 |
|---|---|
| author | Seo, Minhyuk Koh, Hyunseo Jeung, Wonje Lee, Minjae Kim, San Lee, Hankook Cho, Sungjun Choi, Sungik Kim, Hyunwoo Choi, Jonghyun |
| author_facet | Seo, Minhyuk Koh, Hyunseo Jeung, Wonje Lee, Minjae Kim, San Lee, Hankook Cho, Sungjun Choi, Sungik Kim, Hyunwoo Choi, Jonghyun |
| contents | Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we propose an efficient online continual learning method using the neural collapse phenomenon. In particular, we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K, we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaussian scheduled continuous (i.e., boundary-free) data setups. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01628 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Equi-angular Representations for Online Continual Learning Seo, Minhyuk Koh, Hyunseo Jeung, Wonje Lee, Minjae Kim, San Lee, Hankook Cho, Sungjun Choi, Sungik Kim, Hyunwoo Choi, Jonghyun Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we propose an efficient online continual learning method using the neural collapse phenomenon. In particular, we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K, we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaussian scheduled continuous (i.e., boundary-free) data setups. |
| title | Learning Equi-angular Representations for Online Continual Learning |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2404.01628 |