Is Parameter Isolation Better for Prompt-Based Continual Learning?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914288805871616 |
|---|---|
| author | Li, Jiangyang Ding, Chenhao Dong, Songlin Wang, Qiang Zhao, Jianchao He, Yuhang Gong, Yihong |
| author_facet | Li, Jiangyang Ding, Chenhao Dong, Songlin Wang, Qiang Zhao, Jianchao He, Yuhang Gong, Yihong |
| contents | Prompt-based continual learning methods effectively mitigate catastrophic forgetting. However, most existing methods assign a fixed set of prompts to each task, completely isolating knowledge across tasks and resulting in suboptimal parameter utilization. To address this, we consider the practical needs of continual learning and propose a prompt-sharing framework. This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts to achieve dynamic decoupling and collaborative optimization of task-specific feature representations. Furthermore, we introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates, thereby mitigating inefficient parameter usage and knowledge forgetting. Extensive analysis and empirical results demonstrate that our approach consistently outperforms existing static allocation strategies in effectiveness and efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20894 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Is Parameter Isolation Better for Prompt-Based Continual Learning? Li, Jiangyang Ding, Chenhao Dong, Songlin Wang, Qiang Zhao, Jianchao He, Yuhang Gong, Yihong Machine Learning Prompt-based continual learning methods effectively mitigate catastrophic forgetting. However, most existing methods assign a fixed set of prompts to each task, completely isolating knowledge across tasks and resulting in suboptimal parameter utilization. To address this, we consider the practical needs of continual learning and propose a prompt-sharing framework. This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts to achieve dynamic decoupling and collaborative optimization of task-specific feature representations. Furthermore, we introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates, thereby mitigating inefficient parameter usage and knowledge forgetting. Extensive analysis and empirical results demonstrate that our approach consistently outperforms existing static allocation strategies in effectiveness and efficiency. |
| title | Is Parameter Isolation Better for Prompt-Based Continual Learning? |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.20894 |