Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916653048004608 |
|---|---|
| author | Huo, Wei Liu, Changxin Ding, Kemi Johansson, Karl Henrik Shi, Ling |
| author_facet | Huo, Wei Liu, Changxin Ding, Kemi Johansson, Karl Henrik Shi, Ling |
| contents | This paper investigates the use of the cubic-regularized Newton method within a federated learning framework while addressing two major concerns that commonly arise in federated learning: privacy leakage and communication bottleneck. We introduce a federated learning algorithm called Differentially Private Federated Cubic Regularized Newton (DP-FCRN). By leveraging second-order techniques, our algorithm achieves lower iteration complexity compared to first-order methods. We also incorporate noise perturbation during local computations to ensure privacy. Furthermore, we employ sparsification in uplink transmission, which not only reduces the communication costs but also amplifies the privacy guarantee. Specifically, this approach reduces the necessary noise intensity without compromising privacy protection. We analyze the convergence properties of our algorithm and establish the privacy guarantee. Finally, we validate the effectiveness of the proposed algorithm through experiments on a benchmark dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04315 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy Huo, Wei Liu, Changxin Ding, Kemi Johansson, Karl Henrik Shi, Ling Machine Learning Systems and Control This paper investigates the use of the cubic-regularized Newton method within a federated learning framework while addressing two major concerns that commonly arise in federated learning: privacy leakage and communication bottleneck. We introduce a federated learning algorithm called Differentially Private Federated Cubic Regularized Newton (DP-FCRN). By leveraging second-order techniques, our algorithm achieves lower iteration complexity compared to first-order methods. We also incorporate noise perturbation during local computations to ensure privacy. Furthermore, we employ sparsification in uplink transmission, which not only reduces the communication costs but also amplifies the privacy guarantee. Specifically, this approach reduces the necessary noise intensity without compromising privacy protection. We analyze the convergence properties of our algorithm and establish the privacy guarantee. Finally, we validate the effectiveness of the proposed algorithm through experiments on a benchmark dataset. |
| title | Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy |
| topic | Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2408.04315 |