Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.11074 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912461433602048 |
|---|---|
| author | Yin, Baocai Zhao, Ji Jiang, Huajie Hou, Ningning Hu, Yongli Beheshti, Amin Yang, Ming-Hsuan Qi, Yuankai |
| author_facet | Yin, Baocai Zhao, Ji Jiang, Huajie Hou, Ningning Hu, Yongli Beheshti, Amin Yang, Ming-Hsuan Qi, Yuankai |
| contents | Continual learning (CL) enables models to adapt to evolving data streams. A major challenge of CL is catastrophic forgetting, where new knowledge will overwrite previously acquired knowledge. Traditional methods usually retain the past data for replay or add additional branches in the model to learn new knowledge, which has high memory requirements. In this paper, we propose a novel lightweight CL framework, Adapter-Enhanced Semantic Prompting (AESP), which integrates prompt tuning and adapter techniques. Specifically, we design semantic-guided prompts to enhance the generalization ability of visual features and utilize adapters to efficiently fuse the semantic information, aiming to learn more adaptive features for the continual learning task. Furthermore, to choose the right task prompt for feature adaptation, we have developed a novel matching mechanism for prompt selection. Extensive experiments on three CL datasets demonstrate that our approach achieves favorable performance across multiple metrics, showing its potential for advancing CL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11074 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adapter-Enhanced Semantic Prompting for Continual Learning Yin, Baocai Zhao, Ji Jiang, Huajie Hou, Ningning Hu, Yongli Beheshti, Amin Yang, Ming-Hsuan Qi, Yuankai Computer Vision and Pattern Recognition Machine Learning Continual learning (CL) enables models to adapt to evolving data streams. A major challenge of CL is catastrophic forgetting, where new knowledge will overwrite previously acquired knowledge. Traditional methods usually retain the past data for replay or add additional branches in the model to learn new knowledge, which has high memory requirements. In this paper, we propose a novel lightweight CL framework, Adapter-Enhanced Semantic Prompting (AESP), which integrates prompt tuning and adapter techniques. Specifically, we design semantic-guided prompts to enhance the generalization ability of visual features and utilize adapters to efficiently fuse the semantic information, aiming to learn more adaptive features for the continual learning task. Furthermore, to choose the right task prompt for feature adaptation, we have developed a novel matching mechanism for prompt selection. Extensive experiments on three CL datasets demonstrate that our approach achieves favorable performance across multiple metrics, showing its potential for advancing CL. |
| title | Adapter-Enhanced Semantic Prompting for Continual Learning |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2412.11074 |