Continuous Newton-like Methods featuring Inertia and Variable Mass
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913229916078080 |
|---|---|
| author | Castera, Camille Attouch, Hedy Fadili, Jalal Ochs, Peter |
| author_facet | Castera, Camille Attouch, Hedy Fadili, Jalal Ochs, Peter |
| contents | We introduce a new dynamical system, at the interface between second-order dynamics with inertia and Newton's method. This system extends the class of inertial Newton-like dynamics by featuring a time-dependent parameter in front of the acceleration, called variable mass. For strongly convex optimization, we provide guarantees on how the Newtonian and inertial behaviors of the system can be non-asymptotically controlled by means of this variable mass. A connection with the Levenberg--Marquardt (or regularized Newton's) method is also made. We then show the effect of the variable mass on the asymptotic rate of convergence of the dynamics, and in particular, how it can turn the latter into an accelerated Newton method. We provide numerical experiments supporting our findings. This work represents a significant step towards designing new algorithms that benefit from the best of both first- and second-order optimization methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_08726 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Continuous Newton-like Methods featuring Inertia and Variable Mass Castera, Camille Attouch, Hedy Fadili, Jalal Ochs, Peter Optimization and Control Dynamical Systems 37N40, 46N10, 49M15, 65B99, 65K10 We introduce a new dynamical system, at the interface between second-order dynamics with inertia and Newton's method. This system extends the class of inertial Newton-like dynamics by featuring a time-dependent parameter in front of the acceleration, called variable mass. For strongly convex optimization, we provide guarantees on how the Newtonian and inertial behaviors of the system can be non-asymptotically controlled by means of this variable mass. A connection with the Levenberg--Marquardt (or regularized Newton's) method is also made. We then show the effect of the variable mass on the asymptotic rate of convergence of the dynamics, and in particular, how it can turn the latter into an accelerated Newton method. We provide numerical experiments supporting our findings. This work represents a significant step towards designing new algorithms that benefit from the best of both first- and second-order optimization methods. |
| title | Continuous Newton-like Methods featuring Inertia and Variable Mass |
| topic | Optimization and Control Dynamical Systems 37N40, 46N10, 49M15, 65B99, 65K10 |
| url | https://arxiv.org/abs/2301.08726 |