PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913673886302208 |
|---|---|
| author | Feng, Yu Geng, Yangli-ao Zhu, Yifan Han, Zongfu Yu, Xie Xue, Kaiwen Luo, Haoran Sun, Mengyang Zhang, Guangwei Song, Meina |
| author_facet | Feng, Yu Geng, Yangli-ao Zhu, Yifan Han, Zongfu Yu, Xie Xue, Kaiwen Luo, Haoran Sun, Mengyang Zhang, Guangwei Song, Meina |
| contents | Federated learning (FL) has gained widespread attention for its privacy-preserving and collaborative learning capabilities. Due to significant statistical heterogeneity, traditional FL struggles to generalize a shared model across diverse data domains. Personalized federated learning addresses this issue by dividing the model into a globally shared part and a locally private part, with the local model correcting representation biases introduced by the global model. Nevertheless, locally converged parameters more accurately capture domain-specific knowledge, and current methods overlook the potential benefits of these parameters. To address these limitations, we propose PM-MoE architecture. This architecture integrates a mixture of personalized modules and an energy-based personalized modules denoising, enabling each client to select beneficial personalized parameters from other clients. We applied the PM-MoE architecture to nine recent model-split-based personalized federated learning algorithms, achieving performance improvements with minimal additional training. Extensive experiments on six widely adopted datasets and two heterogeneity settings validate the effectiveness of our approach. The source code is available at \url{https://github.com/dannis97500/PM-MOE}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_00354 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning Feng, Yu Geng, Yangli-ao Zhu, Yifan Han, Zongfu Yu, Xie Xue, Kaiwen Luo, Haoran Sun, Mengyang Zhang, Guangwei Song, Meina Machine Learning Artificial Intelligence Cryptography and Security Federated learning (FL) has gained widespread attention for its privacy-preserving and collaborative learning capabilities. Due to significant statistical heterogeneity, traditional FL struggles to generalize a shared model across diverse data domains. Personalized federated learning addresses this issue by dividing the model into a globally shared part and a locally private part, with the local model correcting representation biases introduced by the global model. Nevertheless, locally converged parameters more accurately capture domain-specific knowledge, and current methods overlook the potential benefits of these parameters. To address these limitations, we propose PM-MoE architecture. This architecture integrates a mixture of personalized modules and an energy-based personalized modules denoising, enabling each client to select beneficial personalized parameters from other clients. We applied the PM-MoE architecture to nine recent model-split-based personalized federated learning algorithms, achieving performance improvements with minimal additional training. Extensive experiments on six widely adopted datasets and two heterogeneity settings validate the effectiveness of our approach. The source code is available at \url{https://github.com/dannis97500/PM-MOE}. |
| title | PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning |
| topic | Machine Learning Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2502.00354 |