EXACFS -- A CIL Method to mitigate Catastrophic Forgetting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909519236300800 |
|---|---|
| author | Balasubramanian, S Subramaniam, M Sai Talasu, Sai Sriram P, Yedu Krishna Sai, Manepalli Pranav Phanindra Mukkamala, Ravi Gera, Darshan |
| author_facet | Balasubramanian, S Subramaniam, M Sai Talasu, Sai Sriram P, Yedu Krishna Sai, Manepalli Pranav Phanindra Mukkamala, Ravi Gera, Darshan |
| contents | Deep neural networks (DNNS) excel at learning from static datasets but struggle with continual learning, where data arrives sequentially. Catastrophic forgetting, the phenomenon of forgetting previously learned knowledge, is a primary challenge. This paper introduces EXponentially Averaged Class-wise Feature Significance (EXACFS) to mitigate this issue in the class incremental learning (CIL) setting. By estimating the significance of model features for each learned class using loss gradients, gradually aging the significance through the incremental tasks and preserving the significant features through a distillation loss, EXACFS effectively balances remembering old knowledge (stability) and learning new knowledge (plasticity). Extensive experiments on CIFAR-100 and ImageNet-100 demonstrate EXACFS's superior performance in preserving stability while acquiring plasticity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_23751 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EXACFS -- A CIL Method to mitigate Catastrophic Forgetting Balasubramanian, S Subramaniam, M Sai Talasu, Sai Sriram P, Yedu Krishna Sai, Manepalli Pranav Phanindra Mukkamala, Ravi Gera, Darshan Computer Vision and Pattern Recognition Machine Learning Deep neural networks (DNNS) excel at learning from static datasets but struggle with continual learning, where data arrives sequentially. Catastrophic forgetting, the phenomenon of forgetting previously learned knowledge, is a primary challenge. This paper introduces EXponentially Averaged Class-wise Feature Significance (EXACFS) to mitigate this issue in the class incremental learning (CIL) setting. By estimating the significance of model features for each learned class using loss gradients, gradually aging the significance through the incremental tasks and preserving the significant features through a distillation loss, EXACFS effectively balances remembering old knowledge (stability) and learning new knowledge (plasticity). Extensive experiments on CIFAR-100 and ImageNet-100 demonstrate EXACFS's superior performance in preserving stability while acquiring plasticity. |
| title | EXACFS -- A CIL Method to mitigate Catastrophic Forgetting |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2410.23751 |