Stochastic Push-Pull for Decentralized Nonconvex Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910995133235200 |
|---|---|
| author | You, Runze Pu, Shi |
| author_facet | You, Runze Pu, Shi |
| contents | To understand the convergence behavior of the Push-Pull method for decentralized optimization with stochastic gradients (Stochastic Push-Pull), this paper presents a comprehensive analysis. Specifically, we first clarify the algorithm's underlying assumptions, particularly those regarding the network structure and weight matrices. Then, to establish the convergence rate under smooth nonconvex objectives, we introduce a general analytical framework that not only encompasses a broad class of decentralized optimization algorithms, but also recovers or enhances several state-of-the-art results for distributed stochastic gradient tracking methods. A key highlight is the derivation of a sufficient condition under which the Stochastic Push-Pull algorithm achieves linear speedup, matching the scalability of centralized stochastic gradient methods -- a result not previously reported. Extensive numerical experiments validate our theoretical findings, demonstrating the algorithm's effectiveness and robustness across various decentralized optimization scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_07021 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Stochastic Push-Pull for Decentralized Nonconvex Optimization You, Runze Pu, Shi Optimization and Control To understand the convergence behavior of the Push-Pull method for decentralized optimization with stochastic gradients (Stochastic Push-Pull), this paper presents a comprehensive analysis. Specifically, we first clarify the algorithm's underlying assumptions, particularly those regarding the network structure and weight matrices. Then, to establish the convergence rate under smooth nonconvex objectives, we introduce a general analytical framework that not only encompasses a broad class of decentralized optimization algorithms, but also recovers or enhances several state-of-the-art results for distributed stochastic gradient tracking methods. A key highlight is the derivation of a sufficient condition under which the Stochastic Push-Pull algorithm achieves linear speedup, matching the scalability of centralized stochastic gradient methods -- a result not previously reported. Extensive numerical experiments validate our theoretical findings, demonstrating the algorithm's effectiveness and robustness across various decentralized optimization scenarios. |
| title | Stochastic Push-Pull for Decentralized Nonconvex Optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2506.07021 |