Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2207.07493 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909327963455488 |
|---|---|
| author | Ahn, Seyoung Kim, Soohyeong Kwon, Yongseok Park, Joohan Youn, Jiseung Cho, Sunghyun |
| author_facet | Ahn, Seyoung Kim, Soohyeong Kwon, Yongseok Park, Joohan Youn, Jiseung Cho, Sunghyun |
| contents | In 6G mobile communication systems, various AI-based network functions and applications have been standardized. Federated learning (FL) is adopted as the core learning architecture for 6G systems to avoid privacy leakage from mobile user data. However, in FL, users with non-independent and identically distributed (non-IID) datasets can deteriorate the performance of the global model because the convergence direction of the gradient for each dataset is different, thereby inducing a weight divergence problem. To address this problem, we propose a novel diffusion strategy for machine learning (ML) models (FedDif) to maximize the performance of the global model with non-IID data. FedDif enables the local model to learn different distributions before parameter aggregation by passing the local models through users via device-to-device communication. Furthermore, we theoretically demonstrate that FedDif can circumvent the weight-divergence problem. Based on this theory, we propose a communication-efficient diffusion strategy for ML models that can determine the trade-off between learning performance and communication cost using auction theory. The experimental results show that FedDif improves the top-1 test accuracy by up to 34.89\% and reduces communication costs by 14.6% to a maximum of 63.49%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2207_07493 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data Ahn, Seyoung Kim, Soohyeong Kwon, Yongseok Park, Joohan Youn, Jiseung Cho, Sunghyun Distributed, Parallel, and Cluster Computing Machine Learning 68T05 In 6G mobile communication systems, various AI-based network functions and applications have been standardized. Federated learning (FL) is adopted as the core learning architecture for 6G systems to avoid privacy leakage from mobile user data. However, in FL, users with non-independent and identically distributed (non-IID) datasets can deteriorate the performance of the global model because the convergence direction of the gradient for each dataset is different, thereby inducing a weight divergence problem. To address this problem, we propose a novel diffusion strategy for machine learning (ML) models (FedDif) to maximize the performance of the global model with non-IID data. FedDif enables the local model to learn different distributions before parameter aggregation by passing the local models through users via device-to-device communication. Furthermore, we theoretically demonstrate that FedDif can circumvent the weight-divergence problem. Based on this theory, we propose a communication-efficient diffusion strategy for ML models that can determine the trade-off between learning performance and communication cost using auction theory. The experimental results show that FedDif improves the top-1 test accuracy by up to 34.89\% and reduces communication costs by 14.6% to a maximum of 63.49%. |
| title | Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning 68T05 |
| url | https://arxiv.org/abs/2207.07493 |