FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911691383504896 |
|---|---|
| author | Zhang, Boyang Chen, Xiaobing Zhang, Songyang Zhang, Shuai Zhou, Xiangwei Zhang, Jian Sun, Mingxuan |
| author_facet | Zhang, Boyang Chen, Xiaobing Zhang, Songyang Zhang, Shuai Zhou, Xiangwei Zhang, Jian Sun, Mingxuan |
| contents | Mixture-of-Experts (MoE) models enable scalable neural networks through conditional computation, offering enhanced effectiveness and efficiency for next-generation wireless communications. However, deploying MoE with federated learning (FL) over wireless and IoT edge networks faces two critical challenges: 1) resource-constrained clients cannot store large AI models with full expert sets, and 2) non-IID data distributions cause severe expert load imbalance that degrades model performance. To this end, we propose FLEX-MoE, a federated MoE framework that jointly optimizes expert assignment and load balancing under limited client capacity. Specifically, our approach introduces client-expert fitness scores that quantify expert suitability for local datasets through training feedback, and employs an optimization-based algorithm to maximize client-expert specialization while enforcing balanced expert utilization system-wide. Unlike greedy methods that focus solely on personalization while ignoring load imbalance, FLEX-MoE addresses expert utilization skew, which is particularly severe in heterogeneous edge FL. Our experimental results demonstrate superior accuracy and consistently balanced expert utilization across diverse resource-constrained scenarios for edge computing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_23070 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing Zhang, Boyang Chen, Xiaobing Zhang, Songyang Zhang, Shuai Zhou, Xiangwei Zhang, Jian Sun, Mingxuan Machine Learning Mixture-of-Experts (MoE) models enable scalable neural networks through conditional computation, offering enhanced effectiveness and efficiency for next-generation wireless communications. However, deploying MoE with federated learning (FL) over wireless and IoT edge networks faces two critical challenges: 1) resource-constrained clients cannot store large AI models with full expert sets, and 2) non-IID data distributions cause severe expert load imbalance that degrades model performance. To this end, we propose FLEX-MoE, a federated MoE framework that jointly optimizes expert assignment and load balancing under limited client capacity. Specifically, our approach introduces client-expert fitness scores that quantify expert suitability for local datasets through training feedback, and employs an optimization-based algorithm to maximize client-expert specialization while enforcing balanced expert utilization system-wide. Unlike greedy methods that focus solely on personalization while ignoring load imbalance, FLEX-MoE addresses expert utilization skew, which is particularly severe in heterogeneous edge FL. Our experimental results demonstrate superior accuracy and consistently balanced expert utilization across diverse resource-constrained scenarios for edge computing. |
| title | FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.23070 |