Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.02447 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910965426028544 |
|---|---|
| author | Salami, Riccardo Buzzega, Pietro Mosconi, Matteo Verasani, Mattia Calderara, Simone |
| author_facet | Salami, Riccardo Buzzega, Pietro Mosconi, Matteo Verasani, Mattia Calderara, Simone |
| contents | Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) while safeguarding data privacy. On top of that, Federated Continual Learning (FCL) also accounts for data distribution evolving over time, mirroring the dynamic nature of real-world environments. While previous studies have identified Catastrophic Forgetting and Client Drift as primary causes of performance degradation in FCL, we shed light on the importance of Incremental Bias and Federated Bias, which cause models to prioritize classes that are recently introduced or locally predominant, respectively. Our proposal constrains both biases in the last layer by efficiently finetuning a pre-trained backbone using learnable prompts, resulting in clients that produce less biased representations and more biased classifiers. Therefore, instead of solely relying on parameter aggregation, we leverage generative prototypes to effectively balance the predictions of the global model. Our method significantly improves the current State Of The Art, providing an average increase of +7.8% in accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_02447 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Federated Class-Incremental Learning with Hierarchical Generative Prototypes Salami, Riccardo Buzzega, Pietro Mosconi, Matteo Verasani, Mattia Calderara, Simone Machine Learning Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) while safeguarding data privacy. On top of that, Federated Continual Learning (FCL) also accounts for data distribution evolving over time, mirroring the dynamic nature of real-world environments. While previous studies have identified Catastrophic Forgetting and Client Drift as primary causes of performance degradation in FCL, we shed light on the importance of Incremental Bias and Federated Bias, which cause models to prioritize classes that are recently introduced or locally predominant, respectively. Our proposal constrains both biases in the last layer by efficiently finetuning a pre-trained backbone using learnable prompts, resulting in clients that produce less biased representations and more biased classifiers. Therefore, instead of solely relying on parameter aggregation, we leverage generative prototypes to effectively balance the predictions of the global model. Our method significantly improves the current State Of The Art, providing an average increase of +7.8% in accuracy. |
| title | Federated Class-Incremental Learning with Hierarchical Generative Prototypes |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.02447 |