Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.11286 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910702924464128 |
|---|---|
| author | Li, Jiongcheng |
| author_facet | Li, Jiongcheng |
| contents | Optimization problems, arise in many practical applications, from the view points of both theory and numerical methods. Especially, significant improvement in deep learning training came from the Quasi-Newton methods. Quasi-Newton search directions provide an attractive alternative to Newton's method in that they do not require computation of the Hessian and yet still attain a super linear rate of convergence. In Quasi-Newton method, we require Hessian approximation to satisfy the secant equation. In this paper, the Classical Cauchy-Schwartz Inequality is introduced, then more generalization are proposed. And it is seriously proved that Quasi-Newton method is a steepest descent method under the ellipsoid norm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11286 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Quasi-Newton method of Optimization is proved to be a steepest descent method under the ellipsoid norm Li, Jiongcheng Optimization and Control Optimization problems, arise in many practical applications, from the view points of both theory and numerical methods. Especially, significant improvement in deep learning training came from the Quasi-Newton methods. Quasi-Newton search directions provide an attractive alternative to Newton's method in that they do not require computation of the Hessian and yet still attain a super linear rate of convergence. In Quasi-Newton method, we require Hessian approximation to satisfy the secant equation. In this paper, the Classical Cauchy-Schwartz Inequality is introduced, then more generalization are proposed. And it is seriously proved that Quasi-Newton method is a steepest descent method under the ellipsoid norm. |
| title | Quasi-Newton method of Optimization is proved to be a steepest descent method under the ellipsoid norm |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2411.11286 |