Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.12048 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917728388907008 |
|---|---|
| author | Kim, Doyoung Yoon, Susik Park, Dongmin Lee, Youngjun Song, Hwanjun Bang, Jihwan Lee, Jae-Gil |
| author_facet | Kim, Doyoung Yoon, Susik Park, Dongmin Lee, Youngjun Song, Hwanjun Bang, Jihwan Lee, Jae-Gil |
| contents | In real-world continual learning (CL) scenarios, tasks often exhibit intricate and unpredictable semantic shifts, posing challenges for fixed prompt management strategies which are tailored to only handle semantic shifts of uniform degree (i.e., uniformly mild or uniformly abrupt). To address this limitation, we propose an adaptive prompting approach that effectively accommodates semantic shifts of varying degree where mild and abrupt shifts are mixed. AdaPromptCL employs the assign-and-refine semantic grouping mechanism that dynamically manages prompt groups in accordance with the semantic similarity between tasks, enhancing the quality of grouping through continuous refinement. Our experiment results demonstrate that AdaPromptCL outperforms existing prompting methods by up to 21.3%, especially in the benchmark datasets with diverse semantic shifts between tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_12048 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning Kim, Doyoung Yoon, Susik Park, Dongmin Lee, Youngjun Song, Hwanjun Bang, Jihwan Lee, Jae-Gil Machine Learning In real-world continual learning (CL) scenarios, tasks often exhibit intricate and unpredictable semantic shifts, posing challenges for fixed prompt management strategies which are tailored to only handle semantic shifts of uniform degree (i.e., uniformly mild or uniformly abrupt). To address this limitation, we propose an adaptive prompting approach that effectively accommodates semantic shifts of varying degree where mild and abrupt shifts are mixed. AdaPromptCL employs the assign-and-refine semantic grouping mechanism that dynamically manages prompt groups in accordance with the semantic similarity between tasks, enhancing the quality of grouping through continuous refinement. Our experiment results demonstrate that AdaPromptCL outperforms existing prompting methods by up to 21.3%, especially in the benchmark datasets with diverse semantic shifts between tasks. |
| title | One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2311.12048 |