Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.10677 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910947145154560 |
|---|---|
| author | Pitsiorlas, Ioannis Jamoussi, Nour Kountouris, Marios |
| author_facet | Pitsiorlas, Ioannis Jamoussi, Nour Kountouris, Marios |
| contents | This work introduces a novel methodology for assessing catastrophic forgetting (CF) in continual learning. We propose a new conformal prediction (CP)-based metric, termed the Conformal Prediction Confidence Factor (CPCF), to quantify and evaluate CF effectively. Our framework leverages adaptive CP to estimate forgetting by monitoring the model's confidence on previously learned tasks. This approach provides a dynamic and practical solution for monitoring and measuring CF of previous tasks as new ones are introduced, offering greater suitability for real-world applications. Experimental results on four benchmark datasets demonstrate a strong correlation between CPCF and the accuracy of previous tasks, validating the reliability and interpretability of the proposed metric. Our results highlight the potential of CPCF as a robust and effective tool for assessing and understanding CF in dynamic learning environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10677 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Conformal Predictive Measure for Assessing Catastrophic Forgetting Pitsiorlas, Ioannis Jamoussi, Nour Kountouris, Marios Machine Learning Artificial Intelligence This work introduces a novel methodology for assessing catastrophic forgetting (CF) in continual learning. We propose a new conformal prediction (CP)-based metric, termed the Conformal Prediction Confidence Factor (CPCF), to quantify and evaluate CF effectively. Our framework leverages adaptive CP to estimate forgetting by monitoring the model's confidence on previously learned tasks. This approach provides a dynamic and practical solution for monitoring and measuring CF of previous tasks as new ones are introduced, offering greater suitability for real-world applications. Experimental results on four benchmark datasets demonstrate a strong correlation between CPCF and the accuracy of previous tasks, validating the reliability and interpretability of the proposed metric. Our results highlight the potential of CPCF as a robust and effective tool for assessing and understanding CF in dynamic learning environments. |
| title | A Conformal Predictive Measure for Assessing Catastrophic Forgetting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.10677 |