A Homogeneous Second-Order Descent Method for Nonconvex Optimization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916674249162752 |
|---|---|
| author | Zhang, Chuwen Ge, Dongdong He, Chang Jiang, Bo Jiang, Yuntian Xue, Chenyu Ye, Yinyu |
| author_facet | Zhang, Chuwen Ge, Dongdong He, Chang Jiang, Bo Jiang, Yuntian Xue, Chenyu Ye, Yinyu |
| contents | In this paper, we introduce a Homogeneous Second-Order Descent Method (HSODM) using the homogenized quadratic approximation to the original function. The merit of homogenization is that only the leftmost eigenvector of a gradient-Hessian integrated matrix is computed at each iteration. Therefore, the algorithm is a single-loop method that does not need to switch to other sophisticated algorithms and is easy to implement. We show that HSODM has a global convergence rate of $O(ε^{-3/2})$ to find an $ε$-approximate second-order stationary point, and has a local quadratic convergence rate under the standard assumptions. The numerical results demonstrate the advantage of the proposed method over other second-order methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_08212 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | A Homogeneous Second-Order Descent Method for Nonconvex Optimization Zhang, Chuwen Ge, Dongdong He, Chang Jiang, Bo Jiang, Yuntian Xue, Chenyu Ye, Yinyu Optimization and Control In this paper, we introduce a Homogeneous Second-Order Descent Method (HSODM) using the homogenized quadratic approximation to the original function. The merit of homogenization is that only the leftmost eigenvector of a gradient-Hessian integrated matrix is computed at each iteration. Therefore, the algorithm is a single-loop method that does not need to switch to other sophisticated algorithms and is easy to implement. We show that HSODM has a global convergence rate of $O(ε^{-3/2})$ to find an $ε$-approximate second-order stationary point, and has a local quadratic convergence rate under the standard assumptions. The numerical results demonstrate the advantage of the proposed method over other second-order methods. |
| title | A Homogeneous Second-Order Descent Method for Nonconvex Optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2211.08212 |