A Parameterized Barzilai-Borwein Method via Interpolated Least Squares
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915399254147072 |
|---|---|
| author | Xu, Xin |
| author_facet | Xu, Xin |
| contents | The Barzilai-Borwein (BB) method is an effective gradient descent algorithm for solving unconstrained optimization problems. Based on the observation of two classical BB step sizes, by constructing an interpolated least squares model, we propose a novel class of BB step sizes, each of which still retains the quasi-Newton property, with the original two BB step sizes being their two extreme cases. We present the mathematical principle underlying the adaptive alternating BB (ABB) method. Based on this principle, we develop a class of effective adaptive interpolation parameters. For strictly convex quadratic optimization problems, we establish the R-linear convergence of this new gradient descent method by investigating the evolution pattern of the ratio of the absolute values of the gradient components. Numerical experiments are conducted to illustrate our findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03899 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Parameterized Barzilai-Borwein Method via Interpolated Least Squares Xu, Xin Optimization and Control 90C20, 90C25, 90C30 The Barzilai-Borwein (BB) method is an effective gradient descent algorithm for solving unconstrained optimization problems. Based on the observation of two classical BB step sizes, by constructing an interpolated least squares model, we propose a novel class of BB step sizes, each of which still retains the quasi-Newton property, with the original two BB step sizes being their two extreme cases. We present the mathematical principle underlying the adaptive alternating BB (ABB) method. Based on this principle, we develop a class of effective adaptive interpolation parameters. For strictly convex quadratic optimization problems, we establish the R-linear convergence of this new gradient descent method by investigating the evolution pattern of the ratio of the absolute values of the gradient components. Numerical experiments are conducted to illustrate our findings. |
| title | A Parameterized Barzilai-Borwein Method via Interpolated Least Squares |
| topic | Optimization and Control 90C20, 90C25, 90C30 |
| url | https://arxiv.org/abs/2411.03899 |