Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.23726 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916687215853568 |
|---|---|
| author | Wang, Lina Yuan, Yunsheng Wang, Chunxiao Li, Feng |
| author_facet | Wang, Lina Yuan, Yunsheng Wang, Chunxiao Li, Feng |
| contents | In the paradigm of decentralized learning, a group of agents collaborates to learn a global model using distributed datasets without a central server. However, due to the heterogeneity of the local data across the different agents, learning a robust global model is rather challenging. Moreover, the collaboration of the agents relies on their gradient information exchange, which poses a risk of privacy leakage. In this paper, to address these issues, we propose PDSL, a novel privacy-preserved decentralized stochastic learning algorithm with heterogeneous data distribution. On one hand, we innovate in utilizing the notion of Shapley values such that each agent can precisely measure the contributions of its heterogeneous neighbors to the global learning goal; on the other hand, we leverage the notion of differential privacy to prevent each agent from suffering privacy leakage when it contributes gradient information to its neighbors. We conduct both solid theoretical analysis and extensive experiments to demonstrate the efficacy of our PDSL algorithm in terms of privacy preservation and convergence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_23726 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution Wang, Lina Yuan, Yunsheng Wang, Chunxiao Li, Feng Machine Learning In the paradigm of decentralized learning, a group of agents collaborates to learn a global model using distributed datasets without a central server. However, due to the heterogeneity of the local data across the different agents, learning a robust global model is rather challenging. Moreover, the collaboration of the agents relies on their gradient information exchange, which poses a risk of privacy leakage. In this paper, to address these issues, we propose PDSL, a novel privacy-preserved decentralized stochastic learning algorithm with heterogeneous data distribution. On one hand, we innovate in utilizing the notion of Shapley values such that each agent can precisely measure the contributions of its heterogeneous neighbors to the global learning goal; on the other hand, we leverage the notion of differential privacy to prevent each agent from suffering privacy leakage when it contributes gradient information to its neighbors. We conduct both solid theoretical analysis and extensive experiments to demonstrate the efficacy of our PDSL algorithm in terms of privacy preservation and convergence. |
| title | PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.23726 |