Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910864560357376 |
|---|---|
| author | Tran, Quyen Phan, Hoang Le, Minh Truong, Tuan Phung, Dinh Ngo, Linh Nguyen, Thien Ho, Nhat Le, Trung |
| author_facet | Tran, Quyen Phan, Hoang Le, Minh Truong, Tuan Phung, Dinh Ngo, Linh Nguyen, Thien Ho, Nhat Le, Trung |
| contents | Humans perceive the world as a series of sequential events, which can be hierarchically organized with different levels of abstraction based on conceptual knowledge. Drawing inspiration from human learning behaviors, this work proposes a novel approach to mitigate catastrophic forgetting in Prompt-based Continual Learning models by exploiting the relationships between continuously emerging class data. We find that applying human habits of organizing and connecting information can serve as an efficient strategy when training deep learning models. Specifically, by building a hierarchical tree structure based on the expanding set of labels, we gain fresh insights into the data, identifying groups of similar classes could easily cause confusion. Additionally, we delve deeper into the hidden connections between classes by exploring the original pretrained model's behavior through an optimal transport-based approach. From these insights, we propose a novel regularization loss function that encourages models to focus more on challenging knowledge areas, thereby enhancing overall performance. Experimentally, our method demonstrated significant superiority over the most robust state-of-the-art models on various benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_04327 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning Tran, Quyen Phan, Hoang Le, Minh Truong, Tuan Phung, Dinh Ngo, Linh Nguyen, Thien Ho, Nhat Le, Trung Machine Learning Humans perceive the world as a series of sequential events, which can be hierarchically organized with different levels of abstraction based on conceptual knowledge. Drawing inspiration from human learning behaviors, this work proposes a novel approach to mitigate catastrophic forgetting in Prompt-based Continual Learning models by exploiting the relationships between continuously emerging class data. We find that applying human habits of organizing and connecting information can serve as an efficient strategy when training deep learning models. Specifically, by building a hierarchical tree structure based on the expanding set of labels, we gain fresh insights into the data, identifying groups of similar classes could easily cause confusion. Additionally, we delve deeper into the hidden connections between classes by exploring the original pretrained model's behavior through an optimal transport-based approach. From these insights, we propose a novel regularization loss function that encourages models to focus more on challenging knowledge areas, thereby enhancing overall performance. Experimentally, our method demonstrated significant superiority over the most robust state-of-the-art models on various benchmarks. |
| title | Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.04327 |