Distributed Stochastic Optimization for Non-Smooth and Weakly Convex Problems under Heavy-Tailed Noise
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915287191781376 |
|---|---|
| author | Hu, Jun Sun, Chao Chen, Bo Wang, Jianzheng Wang, Zheming |
| author_facet | Hu, Jun Sun, Chao Chen, Bo Wang, Jianzheng Wang, Zheming |
| contents | In existing distributed stochastic optimization studies, it is usually assumed that the gradient noise has a bounded variance. However, recent research shows that the heavy-tailed noise, which allows an unbounded variance, is closer to practical scenarios in many tasks. Under heavy-tailed noise, traditional optimization methods, such as stochastic gradient descent, may have poor performance and even diverge. Thus, it is of great importance to study distributed stochastic optimization algorithms applicable to the heavy-tailed noise scenario. However, most of the existing distributed algorithms under heavy-tailed noise are developed for convex and smooth problems, which limits their applications. This paper proposes a clipping-based distributed stochastic algorithm under heavy-tailed noise that is suitable for non-smooth and weakly convex problems. The convergence of the proposed algorithm is proven, and the conditions on the parameters are given. A numerical experiment is conducted to demonstrate the effectiveness of the proposed algorithm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_09279 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Distributed Stochastic Optimization for Non-Smooth and Weakly Convex Problems under Heavy-Tailed Noise Hu, Jun Sun, Chao Chen, Bo Wang, Jianzheng Wang, Zheming Optimization and Control In existing distributed stochastic optimization studies, it is usually assumed that the gradient noise has a bounded variance. However, recent research shows that the heavy-tailed noise, which allows an unbounded variance, is closer to practical scenarios in many tasks. Under heavy-tailed noise, traditional optimization methods, such as stochastic gradient descent, may have poor performance and even diverge. Thus, it is of great importance to study distributed stochastic optimization algorithms applicable to the heavy-tailed noise scenario. However, most of the existing distributed algorithms under heavy-tailed noise are developed for convex and smooth problems, which limits their applications. This paper proposes a clipping-based distributed stochastic algorithm under heavy-tailed noise that is suitable for non-smooth and weakly convex problems. The convergence of the proposed algorithm is proven, and the conditions on the parameters are given. A numerical experiment is conducted to demonstrate the effectiveness of the proposed algorithm. |
| title | Distributed Stochastic Optimization for Non-Smooth and Weakly Convex Problems under Heavy-Tailed Noise |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2505.09279 |