High Probability Convergence of Distributed Clipped Stochastic Gradient Descent with Heavy-tailed Noise
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908412431826944 |
|---|---|
| author | Yang, Yuchen Lu, Kaihong Wang, Long |
| author_facet | Yang, Yuchen Lu, Kaihong Wang, Long |
| contents | In this paper, the problem of distributed optimization is studied via a network of agents. Each agent only has access to a noisy gradient of its own objective function, and can communicate with its neighbors via a network. To handle this problem, a distributed clipped stochastic gradient descent algorithm is proposed, and the high probability convergence of the algorithm is studied. Existing works on distributed algorithms involving stochastic gradients only consider the light-tailed noises. Different from them, we study the case with heavy-tailed settings. Under mild assumptions on the graph connectivity, we prove that the algorithm converges in high probability under a certain clipping operator. Finally, a simulation is provided to demonstrate the effectiveness of our theoretical results |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_11647 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | High Probability Convergence of Distributed Clipped Stochastic Gradient Descent with Heavy-tailed Noise Yang, Yuchen Lu, Kaihong Wang, Long Optimization and Control In this paper, the problem of distributed optimization is studied via a network of agents. Each agent only has access to a noisy gradient of its own objective function, and can communicate with its neighbors via a network. To handle this problem, a distributed clipped stochastic gradient descent algorithm is proposed, and the high probability convergence of the algorithm is studied. Existing works on distributed algorithms involving stochastic gradients only consider the light-tailed noises. Different from them, we study the case with heavy-tailed settings. Under mild assumptions on the graph connectivity, we prove that the algorithm converges in high probability under a certain clipping operator. Finally, a simulation is provided to demonstrate the effectiveness of our theoretical results |
| title | High Probability Convergence of Distributed Clipped Stochastic Gradient Descent with Heavy-tailed Noise |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2506.11647 |