FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910513222385664 |
|---|---|
| author | Hu, Ming Zhou, Peiheng Yue, Zhihao Ling, Zhiwei Huang, Yihao Li, Anran Liu, Yang Lian, Xiang Chen, Mingsong |
| author_facet | Hu, Ming Zhou, Peiheng Yue, Zhihao Ling, Zhiwei Huang, Yihao Li, Anran Liu, Yang Lian, Xiang Chen, Mingsong |
| contents | As a promising distributed machine learning paradigm, Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. By adopting the classic one-to-multi training scheme (i.e., FedAvg), where the cloud server dispatches one single global model to multiple involved clients, conventional FL methods can achieve collaborative model training without data sharing. However, since only one global model cannot always accommodate all the incompatible convergence directions of local models, existing FL approaches greatly suffer from inferior classification accuracy. To address this issue, we present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. Unlike traditional FL methods, in each round of FL training, FedCross uses multiple middleware models to conduct weighted fusion individually. Since the middleware models used by FedCross can quickly converge into the same flat valley in terms of loss landscapes, the generated global model can achieve a well-generalization. Experimental results on various well-known datasets show that, compared with state-of-the-art FL methods, FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_08285 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation Hu, Ming Zhou, Peiheng Yue, Zhihao Ling, Zhiwei Huang, Yihao Li, Anran Liu, Yang Lian, Xiang Chen, Mingsong Machine Learning As a promising distributed machine learning paradigm, Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. By adopting the classic one-to-multi training scheme (i.e., FedAvg), where the cloud server dispatches one single global model to multiple involved clients, conventional FL methods can achieve collaborative model training without data sharing. However, since only one global model cannot always accommodate all the incompatible convergence directions of local models, existing FL approaches greatly suffer from inferior classification accuracy. To address this issue, we present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. Unlike traditional FL methods, in each round of FL training, FedCross uses multiple middleware models to conduct weighted fusion individually. Since the middleware models used by FedCross can quickly converge into the same flat valley in terms of loss landscapes, the generated global model can achieve a well-generalization. Experimental results on various well-known datasets show that, compared with state-of-the-art FL methods, FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead. |
| title | FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2210.08285 |