Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2505.07450 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912378767015936 |
|---|---|
| author | De La Fuente, Neil Pilligua, Maria Vidal, Daniel Soutiff, Albin Curreli, Cecilia Cremers, Daniel Barsky, Andrey |
| author_facet | De La Fuente, Neil Pilligua, Maria Vidal, Daniel Soutiff, Albin Curreli, Cecilia Cremers, Daniel Barsky, Andrey |
| contents | Continual learning (CL) aims to learn a sequence of tasks without forgetting prior knowledge, but gradient updates for a new task often overwrite the weights learned earlier, causing catastrophic forgetting (CF). We propose Prototype-Augmented Hypernetworks (PAH), a framework where a single hypernetwork, conditioned on learnable task prototypes, dynamically generates task-specific classifier heads on demand. To mitigate forgetting, PAH combines cross-entropy with dual distillation losses, one to align logits and another to align prototypes, ensuring stable feature representations across tasks. Evaluations on Split-CIFAR100 and TinyImageNet demonstrate that PAH achieves state-of-the-art performance, reaching 74.5 % and 63.7 % accuracy with only 1.7 % and 4.4 % forgetting, respectively, surpassing prior methods without storing samples or heads. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_07450 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Prototype Augmented Hypernetworks for Continual Learning De La Fuente, Neil Pilligua, Maria Vidal, Daniel Soutiff, Albin Curreli, Cecilia Cremers, Daniel Barsky, Andrey Machine Learning Artificial Intelligence Continual learning (CL) aims to learn a sequence of tasks without forgetting prior knowledge, but gradient updates for a new task often overwrite the weights learned earlier, causing catastrophic forgetting (CF). We propose Prototype-Augmented Hypernetworks (PAH), a framework where a single hypernetwork, conditioned on learnable task prototypes, dynamically generates task-specific classifier heads on demand. To mitigate forgetting, PAH combines cross-entropy with dual distillation losses, one to align logits and another to align prototypes, ensuring stable feature representations across tasks. Evaluations on Split-CIFAR100 and TinyImageNet demonstrate that PAH achieves state-of-the-art performance, reaching 74.5 % and 63.7 % accuracy with only 1.7 % and 4.4 % forgetting, respectively, surpassing prior methods without storing samples or heads. |
| title | Prototype Augmented Hypernetworks for Continual Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.07450 |