On Convergence and Stability of Two Extended BB-like Step Sizes
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908492572393472 |
|---|---|
| author | Xu, Xin |
| author_facet | Xu, Xin |
| contents | The Barzilai-Borwein (BB) step sizes have a profound impact on gradient descent methods. In this work, we propose two new gradient step sizes: one longer than the original long BB step size, and the other shorter than the original short BB step size. This extends the bounds of the original BB step sizes. For strictly convex quadratic optimization problems, we employ the dynamics of difference equations to prove that these two new methods achieve R-linear convergence. Regarding stability, we surprisingly find that under certain conditions, the gradient descent method corresponding to the longer step size is stable, whereas the shorter step size consistently leads to instability. Numerical results validate these stability theories. Here, stability refers to whether the gradient norm decreases monotonically. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12731 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On Convergence and Stability of Two Extended BB-like Step Sizes Xu, Xin Optimization and Control 90C25 The Barzilai-Borwein (BB) step sizes have a profound impact on gradient descent methods. In this work, we propose two new gradient step sizes: one longer than the original long BB step size, and the other shorter than the original short BB step size. This extends the bounds of the original BB step sizes. For strictly convex quadratic optimization problems, we employ the dynamics of difference equations to prove that these two new methods achieve R-linear convergence. Regarding stability, we surprisingly find that under certain conditions, the gradient descent method corresponding to the longer step size is stable, whereas the shorter step size consistently leads to instability. Numerical results validate these stability theories. Here, stability refers to whether the gradient norm decreases monotonically. |
| title | On Convergence and Stability of Two Extended BB-like Step Sizes |
| topic | Optimization and Control 90C25 |
| url | https://arxiv.org/abs/2506.12731 |