Adaptive proximal gradient methods are universal without approximation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909242112344064 |
|---|---|
| author | Oikonomidis, Konstantinos A. Laude, Emanuel Latafat, Puya Themelis, Andreas Patrinos, Panagiotis |
| author_facet | Oikonomidis, Konstantinos A. Laude, Emanuel Latafat, Puya Themelis, Andreas Patrinos, Panagiotis |
| contents | We show that adaptive proximal gradient methods for convex problems are not restricted to traditional Lipschitzian assumptions. Our analysis reveals that a class of linesearch-free methods is still convergent under mere local Hölder gradient continuity, covering in particular continuously differentiable semi-algebraic functions. To mitigate the lack of local Lipschitz continuity, popular approaches revolve around $\varepsilon$-oracles and/or linesearch procedures. In contrast, we exploit plain Hölder inequalities not entailing any approximation, all while retaining the linesearch-free nature of adaptive schemes. Furthermore, we prove full sequence convergence without prior knowledge of local Hölder constants nor of the order of Hölder continuity. Numerical experiments make comparisons with baseline methods on diverse tasks from machine learning covering both the locally and the globally Hölder setting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06271 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptive proximal gradient methods are universal without approximation Oikonomidis, Konstantinos A. Laude, Emanuel Latafat, Puya Themelis, Andreas Patrinos, Panagiotis Optimization and Control Machine Learning 65K05, 90C06, 90C25, 90C30, 90C47 We show that adaptive proximal gradient methods for convex problems are not restricted to traditional Lipschitzian assumptions. Our analysis reveals that a class of linesearch-free methods is still convergent under mere local Hölder gradient continuity, covering in particular continuously differentiable semi-algebraic functions. To mitigate the lack of local Lipschitz continuity, popular approaches revolve around $\varepsilon$-oracles and/or linesearch procedures. In contrast, we exploit plain Hölder inequalities not entailing any approximation, all while retaining the linesearch-free nature of adaptive schemes. Furthermore, we prove full sequence convergence without prior knowledge of local Hölder constants nor of the order of Hölder continuity. Numerical experiments make comparisons with baseline methods on diverse tasks from machine learning covering both the locally and the globally Hölder setting. |
| title | Adaptive proximal gradient methods are universal without approximation |
| topic | Optimization and Control Machine Learning 65K05, 90C06, 90C25, 90C30, 90C47 |
| url | https://arxiv.org/abs/2402.06271 |