Accurate Forgetting for Heterogeneous Federated Continual Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915162566426624 |
|---|---|
| author | Wuerkaixi, Abudukelimu Cui, Sen Zhang, Jingfeng Yan, Kunda Han, Bo Niu, Gang Fang, Lei Zhang, Changshui Sugiyama, Masashi |
| author_facet | Wuerkaixi, Abudukelimu Cui, Sen Zhang, Jingfeng Yan, Kunda Han, Bo Niu, Gang Fang, Lei Zhang, Changshui Sugiyama, Masashi |
| contents | Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under-explored. Bridging FL and continual learning (CL) gives rise to a challenging practical problem: federated continual learning (FCL). Existing research in FCL primarily focuses on mitigating the catastrophic forgetting issue of continual learning while collaborating with other clients. We argue that the forgetting phenomena are not invariably detrimental. In this paper, we consider a more practical and challenging FCL setting characterized by potentially unrelated or even antagonistic data/tasks across different clients. In the FL scenario, statistical heterogeneity and data noise among clients may exhibit spurious correlations which result in biased feature learning. While existing CL strategies focus on a complete utilization of previous knowledge, we found that forgetting biased information is beneficial in our study. Therefore, we propose a new concept accurate forgetting (AF) and develop a novel generative-replay method~\method~which selectively utilizes previous knowledge in federated networks. We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge. Comprehensive experiments affirm the superiority of our method over baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14205 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Accurate Forgetting for Heterogeneous Federated Continual Learning Wuerkaixi, Abudukelimu Cui, Sen Zhang, Jingfeng Yan, Kunda Han, Bo Niu, Gang Fang, Lei Zhang, Changshui Sugiyama, Masashi Machine Learning Artificial Intelligence Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under-explored. Bridging FL and continual learning (CL) gives rise to a challenging practical problem: federated continual learning (FCL). Existing research in FCL primarily focuses on mitigating the catastrophic forgetting issue of continual learning while collaborating with other clients. We argue that the forgetting phenomena are not invariably detrimental. In this paper, we consider a more practical and challenging FCL setting characterized by potentially unrelated or even antagonistic data/tasks across different clients. In the FL scenario, statistical heterogeneity and data noise among clients may exhibit spurious correlations which result in biased feature learning. While existing CL strategies focus on a complete utilization of previous knowledge, we found that forgetting biased information is beneficial in our study. Therefore, we propose a new concept accurate forgetting (AF) and develop a novel generative-replay method~\method~which selectively utilizes previous knowledge in federated networks. We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge. Comprehensive experiments affirm the superiority of our method over baselines. |
| title | Accurate Forgetting for Heterogeneous Federated Continual Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.14205 |