On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916317969252352 |
|---|---|
| author | Latafat, Puya Themelis, Andreas Patrinos, Panagiotis |
| author_facet | Latafat, Puya Themelis, Andreas Patrinos, Panagiotis |
| contents | Building upon recent works on linesearch-free adaptive proximal gradient methods, this paper proposes adaPG$^{q,r}$, a framework that unifies and extends existing results by providing larger stepsize policies and improved lower bounds. Different choices of the parameters $q$ and $r$ are discussed and the efficacy of the resulting methods is demonstrated through numerical simulations. In an attempt to better understand the underlying theory, its convergence is established in a more general setting that allows for time-varying parameters. Finally, an adaptive alternating minimization algorithm is presented by exploring the dual setting. This algorithm not only incorporates additional adaptivity, but also expands its applicability beyond standard strongly convex settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_18431 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms Latafat, Puya Themelis, Andreas Patrinos, Panagiotis Optimization and Control Machine Learning 65K05, 90C06, 90C25, 90C30, 90C47 Building upon recent works on linesearch-free adaptive proximal gradient methods, this paper proposes adaPG$^{q,r}$, a framework that unifies and extends existing results by providing larger stepsize policies and improved lower bounds. Different choices of the parameters $q$ and $r$ are discussed and the efficacy of the resulting methods is demonstrated through numerical simulations. In an attempt to better understand the underlying theory, its convergence is established in a more general setting that allows for time-varying parameters. Finally, an adaptive alternating minimization algorithm is presented by exploring the dual setting. This algorithm not only incorporates additional adaptivity, but also expands its applicability beyond standard strongly convex settings. |
| title | On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms |
| topic | Optimization and Control Machine Learning 65K05, 90C06, 90C25, 90C30, 90C47 |
| url | https://arxiv.org/abs/2311.18431 |