Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916571087110144 |
|---|---|
| author | Le, Minh Luu, Tien Ngoc The, An Nguyen Le, Thanh-Thien Nguyen, Trang Nguyen, Tung Thanh Van, Linh Ngo Nguyen, Thien Huu |
| author_facet | Le, Minh Luu, Tien Ngoc The, An Nguyen Le, Thanh-Thien Nguyen, Trang Nguyen, Tung Thanh Van, Linh Ngo Nguyen, Thien Huu |
| contents | To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix-tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_08285 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective Le, Minh Luu, Tien Ngoc The, An Nguyen Le, Thanh-Thien Nguyen, Trang Nguyen, Tung Thanh Van, Linh Ngo Nguyen, Thien Huu Computation and Language Machine Learning To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix-tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction. |
| title | Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2412.08285 |