Quasi-Newton method of Optimization is proved to be a steepest descent method under the ellipsoid norm
Fuente:
arXiv
Enregistré dans:
| Auteur principal: | |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910702924464128 |
|---|---|
| author | Li, Jiongcheng |
| author_facet | Li, Jiongcheng |
| contents | Optimization problems, arise in many practical applications, from the view points of both theory and numerical methods. Especially, significant improvement in deep learning training came from the Quasi-Newton methods. Quasi-Newton search directions provide an attractive alternative to Newton's method in that they do not require computation of the Hessian and yet still attain a super linear rate of convergence. In Quasi-Newton method, we require Hessian approximation to satisfy the secant equation. In this paper, the Classical Cauchy-Schwartz Inequality is introduced, then more generalization are proposed. And it is seriously proved that Quasi-Newton method is a steepest descent method under the ellipsoid norm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11286 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Quasi-Newton method of Optimization is proved to be a steepest descent method under the ellipsoid norm Li, Jiongcheng Optimization and Control Optimization problems, arise in many practical applications, from the view points of both theory and numerical methods. Especially, significant improvement in deep learning training came from the Quasi-Newton methods. Quasi-Newton search directions provide an attractive alternative to Newton's method in that they do not require computation of the Hessian and yet still attain a super linear rate of convergence. In Quasi-Newton method, we require Hessian approximation to satisfy the secant equation. In this paper, the Classical Cauchy-Schwartz Inequality is introduced, then more generalization are proposed. And it is seriously proved that Quasi-Newton method is a steepest descent method under the ellipsoid norm. |
| title | Quasi-Newton method of Optimization is proved to be a steepest descent method under the ellipsoid norm |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2411.11286 |