Semantic-Guided Dynamic Sparsification for Pre-Trained Model-based Class-Incremental Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911449392087040 |
|---|---|
| author | Liu, Ruiqi Diao, Boyu An, Zijia Shao, Runjie An, Zhulin Wang, Fei Xu, Yongjun |
| author_facet | Liu, Ruiqi Diao, Boyu An, Zijia Shao, Runjie An, Zhulin Wang, Fei Xu, Yongjun |
| contents | Class-Incremental Learning (CIL) requires a model to continually learn new classes without forgetting old ones. A common and efficient solution freezes a pre-trained model and employs lightweight adapters, whose parameters are often forced to be orthogonal to prevent inter-task interference. However, we argue that this parameter-constraining method is detrimental to plasticity. To this end, we propose Semantic-Guided Dynamic Sparsification (SGDS), a novel method that proactively guides the activation space by governing the orientation and rank of its subspaces through targeted sparsification. Specifically, SGDS promotes knowledge transfer by encouraging similar classes to share a compact activation subspace, while simultaneously preventing interference by assigning non-overlapping activation subspaces to dissimilar classes. By sculpting class-specific sparse subspaces in the activation space, SGDS effectively mitigates interference without imposing rigid constraints on the parameter space. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of SGDS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_21345 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Semantic-Guided Dynamic Sparsification for Pre-Trained Model-based Class-Incremental Learning Liu, Ruiqi Diao, Boyu An, Zijia Shao, Runjie An, Zhulin Wang, Fei Xu, Yongjun Computer Vision and Pattern Recognition Class-Incremental Learning (CIL) requires a model to continually learn new classes without forgetting old ones. A common and efficient solution freezes a pre-trained model and employs lightweight adapters, whose parameters are often forced to be orthogonal to prevent inter-task interference. However, we argue that this parameter-constraining method is detrimental to plasticity. To this end, we propose Semantic-Guided Dynamic Sparsification (SGDS), a novel method that proactively guides the activation space by governing the orientation and rank of its subspaces through targeted sparsification. Specifically, SGDS promotes knowledge transfer by encouraging similar classes to share a compact activation subspace, while simultaneously preventing interference by assigning non-overlapping activation subspaces to dissimilar classes. By sculpting class-specific sparse subspaces in the activation space, SGDS effectively mitigates interference without imposing rigid constraints on the parameter space. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of SGDS. |
| title | Semantic-Guided Dynamic Sparsification for Pre-Trained Model-based Class-Incremental Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.21345 |