Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2409.15100 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916827818360832 |
|---|---|
| author | Li, Jiaxing Chen, Zihan Chong, Kai Fong Ernest Das, Bikramjit Quek, Tony Q. S. Yang, Howard H. |
| author_facet | Li, Jiaxing Chen, Zihan Chong, Kai Fong Ernest Das, Bikramjit Quek, Tony Q. S. Yang, Howard H. |
| contents | Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15100 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping Li, Jiaxing Chen, Zihan Chong, Kai Fong Ernest Das, Bikramjit Quek, Tony Q. S. Yang, Howard H. Machine Learning Artificial Intelligence Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness. |
| title | Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2409.15100 |