dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908369960304640 |
|---|---|
| author | Xie, Luyuan Luan, Tianyu Cai, Wenyuan Yan, Guochen Chen, Zhaoyu Xi, Nan Fang, Yuejian Shen, Qingni Wu, Zhonghai Yuan, Junsong |
| author_facet | Xie, Luyuan Luan, Tianyu Cai, Wenyuan Yan, Guochen Chen, Zhaoyu Xi, Nan Fang, Yuejian Shen, Qingni Wu, Zhonghai Yuan, Junsong |
| contents | Federated learning has wide applications in the medical field. It enables knowledge sharing among different healthcare institutes while protecting patients' privacy. However, existing federated learning systems are typically centralized, requiring clients to upload client-specific knowledge to a central server for aggregation. This centralized approach would integrate the knowledge from each client into a centralized server, and the knowledge would be already undermined during the centralized integration before it reaches back to each client. Besides, the centralized approach also creates a dependency on the central server, which may affect training stability if the server malfunctions or connections are unstable. To address these issues, we propose a decentralized federated learning framework named dFLMoE. In our framework, clients directly exchange lightweight head models with each other. After exchanging, each client treats both local and received head models as individual experts, and utilizes a client-specific Mixture of Experts (MoE) approach to make collective decisions. This design not only reduces the knowledge damage with client-specific aggregations but also removes the dependency on the central server to enhance the robustness of the framework. We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches under both model homogeneity and heterogeneity settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_10412 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis Xie, Luyuan Luan, Tianyu Cai, Wenyuan Yan, Guochen Chen, Zhaoyu Xi, Nan Fang, Yuejian Shen, Qingni Wu, Zhonghai Yuan, Junsong Machine Learning Artificial Intelligence Federated learning has wide applications in the medical field. It enables knowledge sharing among different healthcare institutes while protecting patients' privacy. However, existing federated learning systems are typically centralized, requiring clients to upload client-specific knowledge to a central server for aggregation. This centralized approach would integrate the knowledge from each client into a centralized server, and the knowledge would be already undermined during the centralized integration before it reaches back to each client. Besides, the centralized approach also creates a dependency on the central server, which may affect training stability if the server malfunctions or connections are unstable. To address these issues, we propose a decentralized federated learning framework named dFLMoE. In our framework, clients directly exchange lightweight head models with each other. After exchanging, each client treats both local and received head models as individual experts, and utilizes a client-specific Mixture of Experts (MoE) approach to make collective decisions. This design not only reduces the knowledge damage with client-specific aggregations but also removes the dependency on the central server to enhance the robustness of the framework. We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches under both model homogeneity and heterogeneity settings. |
| title | dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2503.10412 |