Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914062934212608 |
|---|---|
| author | Zuo, Shiyuan Yan, Xingrun Fan, Rongfei Hu, Han Shan, Hangguan Quek, Tony Q. S. Zhao, Puning |
| author_facet | Zuo, Shiyuan Yan, Xingrun Fan, Rongfei Hu, Han Shan, Hangguan Quek, Tony Q. S. Zhao, Puning |
| contents | This paper addresses federated learning (FL) in the context of malicious Byzantine attacks and data heterogeneity. We introduce a novel Robust Average Gradient Algorithm (RAGA), which uses the geometric median for aggregation and {allows flexible round number for local updates.} Unlike most existing resilient approaches, which base their convergence analysis on strongly-convex loss functions or homogeneously distributed datasets, this work conducts convergence analysis for both strongly-convex and non-convex loss functions over heterogeneous datasets. The theoretical analysis indicates that as long as the fraction of the {data} from malicious users is less than half, RAGA can achieve convergence at a rate of $\mathcal{O}({1}/{T^{2/3- δ}})$ for non-convex loss functions, where $T$ is the iteration number and $δ\in (0, 2/3)$. For strongly-convex loss functions, the convergence rate is linear. Furthermore, the stationary point or global optimal solution is shown to be attainable as data heterogeneity diminishes. Experimental results validate the robustness of RAGA against Byzantine attacks and demonstrate its superior convergence performance compared to baselines under varying intensities of Byzantine attacks on heterogeneous datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13374 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity Zuo, Shiyuan Yan, Xingrun Fan, Rongfei Hu, Han Shan, Hangguan Quek, Tony Q. S. Zhao, Puning Machine Learning Artificial Intelligence Cryptography and Security This paper addresses federated learning (FL) in the context of malicious Byzantine attacks and data heterogeneity. We introduce a novel Robust Average Gradient Algorithm (RAGA), which uses the geometric median for aggregation and {allows flexible round number for local updates.} Unlike most existing resilient approaches, which base their convergence analysis on strongly-convex loss functions or homogeneously distributed datasets, this work conducts convergence analysis for both strongly-convex and non-convex loss functions over heterogeneous datasets. The theoretical analysis indicates that as long as the fraction of the {data} from malicious users is less than half, RAGA can achieve convergence at a rate of $\mathcal{O}({1}/{T^{2/3- δ}})$ for non-convex loss functions, where $T$ is the iteration number and $δ\in (0, 2/3)$. For strongly-convex loss functions, the convergence rate is linear. Furthermore, the stationary point or global optimal solution is shown to be attainable as data heterogeneity diminishes. Experimental results validate the robustness of RAGA against Byzantine attacks and demonstrate its superior convergence performance compared to baselines under varying intensities of Byzantine attacks on heterogeneous datasets. |
| title | Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity |
| topic | Machine Learning Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2403.13374 |