Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916258970075136 |
|---|---|
| author | Wang, Leyuan Xiang, Liuyu Wang, Yunlong Wu, Huijia He, Zhaofeng |
| author_facet | Wang, Leyuan Xiang, Liuyu Wang, Yunlong Wu, Huijia He, Zhaofeng |
| contents | Deep neural networks suffer from catastrophic forgetting when continually learning new concepts. In this paper, we analyze this problem from a data imbalance point of view. We argue that the imbalance between old task and new task data contributes to forgetting of the old tasks. Moreover, the increasing imbalance ratio during incremental learning further aggravates the problem. To address the dynamic imbalance issue, we propose Uniform Prototype Contrastive Learning (UPCL), where uniform and compact features are learned. Specifically, we generate a set of non-learnable uniform prototypes before each task starts. Then we assign these uniform prototypes to each class and guide the feature learning through prototype contrastive learning. We also dynamically adjust the relative margin between old and new classes so that the feature distribution will be maintained balanced and compact. Finally, we demonstrate through extensive experiments that the proposed method achieves state-of-the-art performance on several benchmark datasets including CIFAR100, ImageNet100 and TinyImageNet. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15157 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective Wang, Leyuan Xiang, Liuyu Wang, Yunlong Wu, Huijia He, Zhaofeng Computer Vision and Pattern Recognition Deep neural networks suffer from catastrophic forgetting when continually learning new concepts. In this paper, we analyze this problem from a data imbalance point of view. We argue that the imbalance between old task and new task data contributes to forgetting of the old tasks. Moreover, the increasing imbalance ratio during incremental learning further aggravates the problem. To address the dynamic imbalance issue, we propose Uniform Prototype Contrastive Learning (UPCL), where uniform and compact features are learned. Specifically, we generate a set of non-learnable uniform prototypes before each task starts. Then we assign these uniform prototypes to each class and guide the feature learning through prototype contrastive learning. We also dynamically adjust the relative margin between old and new classes so that the feature distribution will be maintained balanced and compact. Finally, we demonstrate through extensive experiments that the proposed method achieves state-of-the-art performance on several benchmark datasets including CIFAR100, ImageNet100 and TinyImageNet. |
| title | Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.15157 |