Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908771003924480 |
|---|---|
| author | Wang, Menglian Liu, Zhuanghua Luo, Luo |
| author_facet | Wang, Menglian Liu, Zhuanghua Luo, Luo |
| contents | This paper studies decentralized stochastic nonconvex optimization problem over row-stochastic networks. We consider the heavy-tailed gradient noise which is empirically observed in many popular real-world applications. Specifically, we propose a decentralized normalized stochastic gradient descent with Pull-Diag gradient tracking, which achieves approximate stationary points with the optimal sample complexity and the near-optimal communication complexity. We further follow our framework to study the setting of undirected networks, also achieving the nearly tight upper complexity bounds. Moreover, we conduct empirical studies to show the practical superiority of the proposed methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_11435 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise Wang, Menglian Liu, Zhuanghua Luo, Luo Optimization and Control Machine Learning This paper studies decentralized stochastic nonconvex optimization problem over row-stochastic networks. We consider the heavy-tailed gradient noise which is empirically observed in many popular real-world applications. Specifically, we propose a decentralized normalized stochastic gradient descent with Pull-Diag gradient tracking, which achieves approximate stationary points with the optimal sample complexity and the near-optimal communication complexity. We further follow our framework to study the setting of undirected networks, also achieving the nearly tight upper complexity bounds. Moreover, we conduct empirical studies to show the practical superiority of the proposed methods. |
| title | Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2601.11435 |