Loss-Transformation Invariance in the Damped Newton Method
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916979360661504 |
|---|---|
| author | Shestakov, Alexander Bohara, Sushil Horváth, Samuel Takáč, Martin Hanzely, Slavomír |
| author_facet | Shestakov, Alexander Bohara, Sushil Horváth, Samuel Takáč, Martin Hanzely, Slavomír |
| contents | The Newton method is a powerful optimization algorithm, valued for its rapid local convergence and elegant geometric properties. However, its theoretical guarantees are usually limited to convex problems. In this work, we ask whether convexity is truly necessary. We introduce the concept of loss-transformation invariance, showing that damped Newton methods are unaffected by monotone transformations of the loss - apart from a simple rescaling of the step size. This insight allows difficult losses to be replaced with easier transformed versions, enabling convexification of many nonconvex problems while preserving the same sequence of iterates. Our analysis also explains the effectiveness of unconventional stepsizes in Newton's method, including values greater than one and even negative steps. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25782 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Loss-Transformation Invariance in the Damped Newton Method Shestakov, Alexander Bohara, Sushil Horváth, Samuel Takáč, Martin Hanzely, Slavomír Optimization and Control The Newton method is a powerful optimization algorithm, valued for its rapid local convergence and elegant geometric properties. However, its theoretical guarantees are usually limited to convex problems. In this work, we ask whether convexity is truly necessary. We introduce the concept of loss-transformation invariance, showing that damped Newton methods are unaffected by monotone transformations of the loss - apart from a simple rescaling of the step size. This insight allows difficult losses to be replaced with easier transformed versions, enabling convexification of many nonconvex problems while preserving the same sequence of iterates. Our analysis also explains the effectiveness of unconventional stepsizes in Newton's method, including values greater than one and even negative steps. |
| title | Loss-Transformation Invariance in the Damped Newton Method |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2509.25782 |