The Randomized Block Coordinate Descent Method in the Hölder Smooth Setting
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929274064207872 |
|---|---|
| author | Maia, Leandro Farias Gutman, David Huckleberry |
| author_facet | Maia, Leandro Farias Gutman, David Huckleberry |
| contents | This work provides the first convergence analysis for the Randomized Block Coordinate Descent method for minimizing a function that is both Hölder smooth and block Hölder smooth. Our analysis applies to objective functions that are non-convex, convex, and strongly convex. For non-convex functions, we show that the expected gradient norm reduces at an $O\left(k^{\fracγ{1+γ}}\right)$ rate, where $k$ is the iteration count and $γ$ is the Hölder exponent. For convex functions, we show that the expected suboptimality gap reduces at the rate $O\left(k^{-γ}\right)$. In the strongly convex setting, we show this rate for the expected suboptimality gap improves to $O\left(k^{-\frac{2γ}{1-γ}}\right)$ when $γ>1$ and to a linear rate when $γ=1$. Notably, these new convergence rates coincide with those furnished in the existing literature for the Lipschitz smooth setting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_08080 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Randomized Block Coordinate Descent Method in the Hölder Smooth Setting Maia, Leandro Farias Gutman, David Huckleberry Optimization and Control This work provides the first convergence analysis for the Randomized Block Coordinate Descent method for minimizing a function that is both Hölder smooth and block Hölder smooth. Our analysis applies to objective functions that are non-convex, convex, and strongly convex. For non-convex functions, we show that the expected gradient norm reduces at an $O\left(k^{\fracγ{1+γ}}\right)$ rate, where $k$ is the iteration count and $γ$ is the Hölder exponent. For convex functions, we show that the expected suboptimality gap reduces at the rate $O\left(k^{-γ}\right)$. In the strongly convex setting, we show this rate for the expected suboptimality gap improves to $O\left(k^{-\frac{2γ}{1-γ}}\right)$ when $γ>1$ and to a linear rate when $γ=1$. Notably, these new convergence rates coincide with those furnished in the existing literature for the Lipschitz smooth setting. |
| title | The Randomized Block Coordinate Descent Method in the Hölder Smooth Setting |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2403.08080 |