Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911837437558784 |
|---|---|
| author | Wang, Liwei Li, Jun Chen, Wen Wu, Qingqing Ding, Ming |
| author_facet | Wang, Liwei Li, Jun Chen, Wen Wu, Qingqing Ding, Ming |
| contents | Federated Learning (FL) facilitates collaborative machine learning by training models on local datasets, and subsequently aggregating these local models at a central server. However, the frequent exchange of model parameters between clients and the central server can result in significant communication overhead during the FL training process. To solve this problem, this paper proposes a novel FL framework, the Model Aggregation with Layer Divergence Feedback mechanism (FedLDF). Specifically, we calculate model divergence between the local model and the global model from the previous round. Then through model layer divergence feedback, the distinct layers of each client are uploaded and the amount of data transferred is reduced effectively. Moreover, the convergence bound reveals that the access ratio of clients has a positive correlation with model performance. Simulation results show that our algorithm uploads local models with reduced communication overhead while upholding a superior global model performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_08324 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning Wang, Liwei Li, Jun Chen, Wen Wu, Qingqing Ding, Ming Distributed, Parallel, and Cluster Computing Federated Learning (FL) facilitates collaborative machine learning by training models on local datasets, and subsequently aggregating these local models at a central server. However, the frequent exchange of model parameters between clients and the central server can result in significant communication overhead during the FL training process. To solve this problem, this paper proposes a novel FL framework, the Model Aggregation with Layer Divergence Feedback mechanism (FedLDF). Specifically, we calculate model divergence between the local model and the global model from the previous round. Then through model layer divergence feedback, the distinct layers of each client are uploaded and the amount of data transferred is reduced effectively. Moreover, the convergence bound reveals that the access ratio of clients has a positive correlation with model performance. Simulation results show that our algorithm uploads local models with reduced communication overhead while upholding a superior global model performance. |
| title | Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2404.08324 |