An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914477599883264 |
|---|---|
| author | Tran, Quyen Nguyen, Hai Phan, Hoang Dao, Quan Ngo, Linh Than, Khoat Phung, Dinh Metaxas, Dimitris Le, Trung |
| author_facet | Tran, Quyen Nguyen, Hai Phan, Hoang Dao, Quan Ngo, Linh Than, Khoat Phung, Dinh Metaxas, Dimitris Le, Trung |
| contents | In online incremental learning, data continuously arrives with substantial distributional shifts, creating a significant challenge because previous samples have limited replay value when learning a new task. Prior research has typically relied on either a single adaptive centroid or multiple fixed centroids to represent each class in the latent space. However, such methods struggle when class data streams are inherently multimodal and require continual centroid updates. To overcome this, we introduce an online Mixture Model learning framework grounded in Optimal Transport theory (MMOT), where centroids evolve incrementally with new data. This approach offers two main advantages: (i) it provides a more precise characterization of complex data streams, and (ii) it enables improved class similarity estimation for unseen samples during inference through MMOT-derived centroids. Furthermore, to strengthen representation learning and mitigate catastrophic forgetting, we design a Dynamic Preservation strategy that regulates the latent space and maintains class separability over time. Experimental evaluations on benchmark datasets confirm the superior effectiveness of our proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_16780 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning Tran, Quyen Nguyen, Hai Phan, Hoang Dao, Quan Ngo, Linh Than, Khoat Phung, Dinh Metaxas, Dimitris Le, Trung Machine Learning Computer Vision and Pattern Recognition In online incremental learning, data continuously arrives with substantial distributional shifts, creating a significant challenge because previous samples have limited replay value when learning a new task. Prior research has typically relied on either a single adaptive centroid or multiple fixed centroids to represent each class in the latent space. However, such methods struggle when class data streams are inherently multimodal and require continual centroid updates. To overcome this, we introduce an online Mixture Model learning framework grounded in Optimal Transport theory (MMOT), where centroids evolve incrementally with new data. This approach offers two main advantages: (i) it provides a more precise characterization of complex data streams, and (ii) it enables improved class similarity estimation for unseen samples during inference through MMOT-derived centroids. Furthermore, to strengthen representation learning and mitigate catastrophic forgetting, we design a Dynamic Preservation strategy that regulates the latent space and maintains class separability over time. Experimental evaluations on benchmark datasets confirm the superior effectiveness of our proposed method. |
| title | An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2211.16780 |