CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910551723999232 |
|---|---|
| author | Feng, Yu Tian, Zhen Zhu, Yifan Han, Zongfu Luo, Haoran Zhang, Guangwei Song, Meina |
| author_facet | Feng, Yu Tian, Zhen Zhu, Yifan Han, Zongfu Luo, Haoran Zhang, Guangwei Song, Meina |
| contents | The key challenge of cross-modal domain-incremental learning (DIL) is to enable the learning model to continuously learn from novel data with different feature distributions under the same task without forgetting old ones. However, existing top-performing methods still cause high forgetting rates, by lacking intra-domain knowledge extraction and inter-domain common prompting strategy. In this paper, we propose a simple yet effective framework, CP-Prompt, by training limited parameters to instruct a pre-trained model to learn new domains and avoid forgetting existing feature distributions. CP-Prompt captures intra-domain knowledge by compositionally inserting personalized prompts on multi-head self-attention layers and then learns the inter-domain knowledge with a common prompting strategy. CP-Prompt shows superiority compared with state-of-the-art baselines among three widely evaluated DIL tasks. The source code is available at https://github.com/dannis97500/CP_Prompt. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_21043 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning Feng, Yu Tian, Zhen Zhu, Yifan Han, Zongfu Luo, Haoran Zhang, Guangwei Song, Meina Computation and Language Artificial Intelligence Machine Learning The key challenge of cross-modal domain-incremental learning (DIL) is to enable the learning model to continuously learn from novel data with different feature distributions under the same task without forgetting old ones. However, existing top-performing methods still cause high forgetting rates, by lacking intra-domain knowledge extraction and inter-domain common prompting strategy. In this paper, we propose a simple yet effective framework, CP-Prompt, by training limited parameters to instruct a pre-trained model to learn new domains and avoid forgetting existing feature distributions. CP-Prompt captures intra-domain knowledge by compositionally inserting personalized prompts on multi-head self-attention layers and then learns the inter-domain knowledge with a common prompting strategy. CP-Prompt shows superiority compared with state-of-the-art baselines among three widely evaluated DIL tasks. The source code is available at https://github.com/dannis97500/CP_Prompt. |
| title | CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.21043 |