Decentralized Conjugate Gradient and Memoryless BFGS Methods
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917894383730688 |
|---|---|
| author | Wang, Liping Wu, Hao Zhang, Hongchao |
| author_facet | Wang, Liping Wu, Hao Zhang, Hongchao |
| contents | This paper proposes a new decentralized conjugate gradient (NDCG) method and a decentralized memoryless BFGS (DMBFGS) method for the nonconvex and strongly convex decentralized optimization problem, respectively, of minimizing a finite sum of continuously differentiable functions over a fixed-connected undirected network. Gradient tracking techniques are applied in these two methods to enhance their convergence properties and the numerical stability. In particular, we show global convergence of NDCG with constant stepsize for general nonconvex smooth decentralized optimization. Our new DMBFGS method uses a scaled memoryless BFGS technique and only requires gradient information to approximate second-order information of the component functions in the objective. We also establish global convergence and linear convergence rate of DMBFGS with constant stepsize for strongly convex smooth decentralized optimization. Our numerical results show that NDCG and DMBFGS are very efficient in terms of both iteration and communication cost compared with other state-of-the-art methods for solving smooth decentralized optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07122 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Decentralized Conjugate Gradient and Memoryless BFGS Methods Wang, Liping Wu, Hao Zhang, Hongchao Optimization and Control This paper proposes a new decentralized conjugate gradient (NDCG) method and a decentralized memoryless BFGS (DMBFGS) method for the nonconvex and strongly convex decentralized optimization problem, respectively, of minimizing a finite sum of continuously differentiable functions over a fixed-connected undirected network. Gradient tracking techniques are applied in these two methods to enhance their convergence properties and the numerical stability. In particular, we show global convergence of NDCG with constant stepsize for general nonconvex smooth decentralized optimization. Our new DMBFGS method uses a scaled memoryless BFGS technique and only requires gradient information to approximate second-order information of the component functions in the objective. We also establish global convergence and linear convergence rate of DMBFGS with constant stepsize for strongly convex smooth decentralized optimization. Our numerical results show that NDCG and DMBFGS are very efficient in terms of both iteration and communication cost compared with other state-of-the-art methods for solving smooth decentralized optimization. |
| title | Decentralized Conjugate Gradient and Memoryless BFGS Methods |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2409.07122 |