A Robust Compressed Push-Pull Method for Decentralized Nonconvex Optimization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910554598146048 |
|---|---|
| author | Liao, Yiwei Li, Zhuorui Pu, Shi Chang, Tsung-Hui |
| author_facet | Liao, Yiwei Li, Zhuorui Pu, Shi Chang, Tsung-Hui |
| contents | In the modern paradigm of multi-agent networks, communication has become one of the main bottlenecks for decentralized optimization, where a large number of agents are involved in minimizing the average of the local cost functions. In this paper, we propose a robust compressed push-pull algorithm (RCPP) that combines gradient tracking with communication compression. In particular, RCPP is robust under a much more general class of compression operators that allow both relative and absolute compression errors, in contrast to the existing works which can handle either one of them or assume convex problems. We show that RCPP enjoys sublinear convergence rate for smooth and possibly nonconvex objective functions over general directed networks. Moreover, under the additional Polyak-Łojasiewicz condition, linear convergence rate can be achieved for RCPP. Numerical examples verify the theoretical findings and demonstrate the efficiency, flexibility, and robustness of the proposed algorithm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01727 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Robust Compressed Push-Pull Method for Decentralized Nonconvex Optimization Liao, Yiwei Li, Zhuorui Pu, Shi Chang, Tsung-Hui Optimization and Control In the modern paradigm of multi-agent networks, communication has become one of the main bottlenecks for decentralized optimization, where a large number of agents are involved in minimizing the average of the local cost functions. In this paper, we propose a robust compressed push-pull algorithm (RCPP) that combines gradient tracking with communication compression. In particular, RCPP is robust under a much more general class of compression operators that allow both relative and absolute compression errors, in contrast to the existing works which can handle either one of them or assume convex problems. We show that RCPP enjoys sublinear convergence rate for smooth and possibly nonconvex objective functions over general directed networks. Moreover, under the additional Polyak-Łojasiewicz condition, linear convergence rate can be achieved for RCPP. Numerical examples verify the theoretical findings and demonstrate the efficiency, flexibility, and robustness of the proposed algorithm. |
| title | A Robust Compressed Push-Pull Method for Decentralized Nonconvex Optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2408.01727 |