Non-asymptotic Global Convergence Analysis of BFGS with the Armijo-Wolfe Line Search
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909450735976448 |
|---|---|
| author | Jin, Qiujiang Jiang, Ruichen Mokhtari, Aryan |
| author_facet | Jin, Qiujiang Jiang, Ruichen Mokhtari, Aryan |
| contents | In this paper, we present the first explicit and non-asymptotic global convergence rates of the BFGS method when implemented with an inexact line search scheme satisfying the Armijo-Wolfe conditions. We show that BFGS achieves a global linear convergence rate of $(1 - \frac{1}κ)^t$ for $μ$-strongly convex functions with $L$-Lipschitz gradients, where $κ= \frac{L}μ$ represents the condition number. Additionally, if the objective function's Hessian is Lipschitz, BFGS with the Armijo-Wolfe line search achieves a linear convergence rate that depends solely on the line search parameters, independent of the condition number. We also establish a global superlinear convergence rate of $\mathcal{O}((\frac{1}{t})^t)$. These global bounds are all valid for any starting point $x_0$ and any symmetric positive definite initial Hessian approximation matrix $B_0$, though the choice of $B_0$ impacts the number of iterations needed to achieve these rates. By synthesizing these results, we outline the first global complexity characterization of BFGS with the Armijo-Wolfe line search. Additionally, we clearly define a mechanism for selecting the step size to satisfy the Armijo-Wolfe conditions and characterize its overall complexity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_16731 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Non-asymptotic Global Convergence Analysis of BFGS with the Armijo-Wolfe Line Search Jin, Qiujiang Jiang, Ruichen Mokhtari, Aryan Optimization and Control In this paper, we present the first explicit and non-asymptotic global convergence rates of the BFGS method when implemented with an inexact line search scheme satisfying the Armijo-Wolfe conditions. We show that BFGS achieves a global linear convergence rate of $(1 - \frac{1}κ)^t$ for $μ$-strongly convex functions with $L$-Lipschitz gradients, where $κ= \frac{L}μ$ represents the condition number. Additionally, if the objective function's Hessian is Lipschitz, BFGS with the Armijo-Wolfe line search achieves a linear convergence rate that depends solely on the line search parameters, independent of the condition number. We also establish a global superlinear convergence rate of $\mathcal{O}((\frac{1}{t})^t)$. These global bounds are all valid for any starting point $x_0$ and any symmetric positive definite initial Hessian approximation matrix $B_0$, though the choice of $B_0$ impacts the number of iterations needed to achieve these rates. By synthesizing these results, we outline the first global complexity characterization of BFGS with the Armijo-Wolfe line search. Additionally, we clearly define a mechanism for selecting the step size to satisfy the Armijo-Wolfe conditions and characterize its overall complexity. |
| title | Non-asymptotic Global Convergence Analysis of BFGS with the Armijo-Wolfe Line Search |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2404.16731 |