Fast convex optimization via closed-loop time scaling of gradient dynamics
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910799687057408 |
|---|---|
| author | Attouch, Hedy Bot, Radu Ioan Nguyen, Dang-Khoa |
| author_facet | Attouch, Hedy Bot, Radu Ioan Nguyen, Dang-Khoa |
| contents | In a Hilbert setting, for convex differentiable optimization, we develop a general framework for adaptive accelerated gradient methods. They are based on damped inertial dynamics where the coefficients are designed in a closed-loop way.
Specifically, the damping is a feedback control of the velocity, or of the gradient of the objective function. For this, we develop a closed-loop version of the time scaling and averaging technique introduced by the authors. We thus obtain autonomous inertial dynamics which involve vanishing viscous damping and implicit Hessian driven damping. By simply using the convergence rates for the continuous steepest descent and Jensen's inequality, without the need for further Lyapunov analysis, we show that the trajectories have several remarkable properties at once: they ensure fast convergence of values, fast convergence of the gradients towards zero, and they converge to optimal solutions. Our approach leads to parallel algorithmic results, that we study in the case of proximal algorithms. These are among the very first general results of this type obtained using autonomous dynamics. Since the proposed numerical methods are based on proximal techniques, the results can be extended to a broader class, specifically to the problem of minimizing a proper, lower semicontinuous, and convex function. Numerical experiments are conducted to demonstrate the efficiency of the proposed methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_00701 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Fast convex optimization via closed-loop time scaling of gradient dynamics Attouch, Hedy Bot, Radu Ioan Nguyen, Dang-Khoa Optimization and Control 37N40, 46N10, 49M30, 65B99, 65K05, 65K10, 90B50, 90C25 In a Hilbert setting, for convex differentiable optimization, we develop a general framework for adaptive accelerated gradient methods. They are based on damped inertial dynamics where the coefficients are designed in a closed-loop way. Specifically, the damping is a feedback control of the velocity, or of the gradient of the objective function. For this, we develop a closed-loop version of the time scaling and averaging technique introduced by the authors. We thus obtain autonomous inertial dynamics which involve vanishing viscous damping and implicit Hessian driven damping. By simply using the convergence rates for the continuous steepest descent and Jensen's inequality, without the need for further Lyapunov analysis, we show that the trajectories have several remarkable properties at once: they ensure fast convergence of values, fast convergence of the gradients towards zero, and they converge to optimal solutions. Our approach leads to parallel algorithmic results, that we study in the case of proximal algorithms. These are among the very first general results of this type obtained using autonomous dynamics. Since the proposed numerical methods are based on proximal techniques, the results can be extended to a broader class, specifically to the problem of minimizing a proper, lower semicontinuous, and convex function. Numerical experiments are conducted to demonstrate the efficiency of the proposed methods. |
| title | Fast convex optimization via closed-loop time scaling of gradient dynamics |
| topic | Optimization and Control 37N40, 46N10, 49M30, 65B99, 65K05, 65K10, 90B50, 90C25 |
| url | https://arxiv.org/abs/2301.00701 |