CLA: Latent Alignment for Online Continual Self-Supervised Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912484030414848 |
|---|---|
| author | Cignoni, Giacomo Cossu, Andrea Gomez-Villa, Alexandra van de Weijer, Joost Carta, Antonio |
| author_facet | Cignoni, Giacomo Cossu, Andrea Gomez-Villa, Alexandra van de Weijer, Joost Carta, Antonio |
| contents | Self-supervised learning (SSL) is able to build latent representations that generalize well to unseen data. However, only a few SSL techniques exist for the online CL setting, where data arrives in small minibatches, the model must comply with a fixed computational budget, and task boundaries are absent. We introduce Continual Latent Alignment (CLA), a novel SSL strategy for Online CL that aligns the representations learned by the current model with past representations to mitigate forgetting. We found that our CLA is able to speed up the convergence of the training process in the online scenario, outperforming state-of-the-art approaches under the same computational budget. Surprisingly, we also discovered that using CLA as a pretraining protocol in the early stages of pretraining leads to a better final performance when compared to a full i.i.d. pretraining. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_10434 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CLA: Latent Alignment for Online Continual Self-Supervised Learning Cignoni, Giacomo Cossu, Andrea Gomez-Villa, Alexandra van de Weijer, Joost Carta, Antonio Machine Learning Computer Vision and Pattern Recognition Self-supervised learning (SSL) is able to build latent representations that generalize well to unseen data. However, only a few SSL techniques exist for the online CL setting, where data arrives in small minibatches, the model must comply with a fixed computational budget, and task boundaries are absent. We introduce Continual Latent Alignment (CLA), a novel SSL strategy for Online CL that aligns the representations learned by the current model with past representations to mitigate forgetting. We found that our CLA is able to speed up the convergence of the training process in the online scenario, outperforming state-of-the-art approaches under the same computational budget. Surprisingly, we also discovered that using CLA as a pretraining protocol in the early stages of pretraining leads to a better final performance when compared to a full i.i.d. pretraining. |
| title | CLA: Latent Alignment for Online Continual Self-Supervised Learning |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.10434 |