Adaptive Regularized Newton Method with Inexact Hessian
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912755720650752 |
|---|---|
| author | Shestakov, Aleksandr Bashirov, Nail Semenov, Andrei Gasnikov, Alexander Takáč, Martin Beznosikov, Aleksandr Kamzolov, Dmitry |
| author_facet | Shestakov, Aleksandr Bashirov, Nail Semenov, Andrei Gasnikov, Alexander Takáč, Martin Beznosikov, Aleksandr Kamzolov, Dmitry |
| contents | Newton's method is the most widespread high-order method, demanding the gradient and the Hessian of the objective function. However, one of the main disadvantages of Newtons method is its lack of global convergence and high iteration cost. Both these drawbacks are critical for modern optimization motivated primarily by current applications in machine learning. In this paper, we introduce a novel algorithm to deal with these disadvantages. Our method can be implemented with various Hessian approximations, including methods that use only the first-order information. Thus, computational costs might be drastically reduced. Also, it can be adjusted to problems' geometries via the usage of different Bregman divergences. The proposed method converges for nonconvex and convex problems globally and it has the same rates as other well-known methods that lack mentioned properties. We present experiments validating our method performs according to the theoretical bounds and shows competitive performance among other Newton-based methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08775 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adaptive Regularized Newton Method with Inexact Hessian Shestakov, Aleksandr Bashirov, Nail Semenov, Andrei Gasnikov, Alexander Takáč, Martin Beznosikov, Aleksandr Kamzolov, Dmitry Optimization and Control Newton's method is the most widespread high-order method, demanding the gradient and the Hessian of the objective function. However, one of the main disadvantages of Newtons method is its lack of global convergence and high iteration cost. Both these drawbacks are critical for modern optimization motivated primarily by current applications in machine learning. In this paper, we introduce a novel algorithm to deal with these disadvantages. Our method can be implemented with various Hessian approximations, including methods that use only the first-order information. Thus, computational costs might be drastically reduced. Also, it can be adjusted to problems' geometries via the usage of different Bregman divergences. The proposed method converges for nonconvex and convex problems globally and it has the same rates as other well-known methods that lack mentioned properties. We present experiments validating our method performs according to the theoretical bounds and shows competitive performance among other Newton-based methods. |
| title | Adaptive Regularized Newton Method with Inexact Hessian |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2512.08775 |