KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911845903761408 |
|---|---|
| author | Xia, Zeke Hu, Ming Yan, Dengke Liu, Ruixuan Li, Anran Xie, Xiaofei Chen, Mingsong |
| author_facet | Xia, Zeke Hu, Ming Yan, Dengke Liu, Ruixuan Li, Anran Xie, Xiaofei Chen, Mingsong |
| contents | Although Split Federated Learning (SFL) is good at enabling knowledge sharing among resource-constrained clients, it suffers from the problem of low training accuracy due to the neglect of data heterogeneity and catastrophic forgetting. To address this issue, we propose a novel SFL approach named KoReA-SFL, which adopts a multi-model aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and a knowledge replay strategy to deal with catastrophic forgetting. Specifically, in KoReA-SFL cloud servers (i.e., fed server and main server) maintain multiple branch model portions rather than a global portion for local training and an aggregated master-model portion for knowledge sharing among branch portions. To avoid catastrophic forgetting, the main server of KoReA-SFL selects multiple assistant devices for knowledge replay according to the training data distribution of each server-side branch-model portion. Experimental results obtained from non-IID and IID scenarios demonstrate that KoReA-SFL significantly outperforms conventional SFL methods (by up to 23.25\% test accuracy improvement). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_12846 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting Xia, Zeke Hu, Ming Yan, Dengke Liu, Ruixuan Li, Anran Xie, Xiaofei Chen, Mingsong Machine Learning Although Split Federated Learning (SFL) is good at enabling knowledge sharing among resource-constrained clients, it suffers from the problem of low training accuracy due to the neglect of data heterogeneity and catastrophic forgetting. To address this issue, we propose a novel SFL approach named KoReA-SFL, which adopts a multi-model aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and a knowledge replay strategy to deal with catastrophic forgetting. Specifically, in KoReA-SFL cloud servers (i.e., fed server and main server) maintain multiple branch model portions rather than a global portion for local training and an aggregated master-model portion for knowledge sharing among branch portions. To avoid catastrophic forgetting, the main server of KoReA-SFL selects multiple assistant devices for knowledge replay according to the training data distribution of each server-side branch-model portion. Experimental results obtained from non-IID and IID scenarios demonstrate that KoReA-SFL significantly outperforms conventional SFL methods (by up to 23.25\% test accuracy improvement). |
| title | KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2404.12846 |