Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910610695913472 |
|---|---|
| author | Gupta, Shuvomoy Das Freund, Robert M. Sun, Xu Andy Taylor, Adrien |
| author_facet | Gupta, Shuvomoy Das Freund, Robert M. Sun, Xu Andy Taylor, Adrien |
| contents | We propose a computer-assisted approach to the analysis of the worst-case convergence of nonlinear conjugate gradient methods (NCGMs). Those methods are known for their generally good empirical performances for large-scale optimization, while having relatively incomplete analyses. Using our computer-assisted approach, we establish novel complexity bounds for the Polak-Ribière-Polyak (PRP) and the Fletcher-Reeves (FR) NCGMs for smooth strongly convex minimization. In particular, we construct mathematical proofs that establish the first non-asymptotic convergence bound for FR (which is historically the first developed NCGM), and a much improved non-asymptotic convergence bound for PRP. Additionally, we provide simple adversarial examples on which these methods do not perform better than gradient descent with exact line search, leaving very little room for improvements on the same class of problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_01530 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses Gupta, Shuvomoy Das Freund, Robert M. Sun, Xu Andy Taylor, Adrien Optimization and Control We propose a computer-assisted approach to the analysis of the worst-case convergence of nonlinear conjugate gradient methods (NCGMs). Those methods are known for their generally good empirical performances for large-scale optimization, while having relatively incomplete analyses. Using our computer-assisted approach, we establish novel complexity bounds for the Polak-Ribière-Polyak (PRP) and the Fletcher-Reeves (FR) NCGMs for smooth strongly convex minimization. In particular, we construct mathematical proofs that establish the first non-asymptotic convergence bound for FR (which is historically the first developed NCGM), and a much improved non-asymptotic convergence bound for PRP. Additionally, we provide simple adversarial examples on which these methods do not perform better than gradient descent with exact line search, leaving very little room for improvements on the same class of problems. |
| title | Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2301.01530 |