Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916627108331520 |
|---|---|
| author | Yu, Chenhao Hong, Yusu Lin, Junhong |
| author_facet | Yu, Chenhao Hong, Yusu Lin, Junhong |
| contents | We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance assumptions for the stochastic gradient noise, we establish high-probability convergence rates of order $\tilde{O}\left(\sqrt{\log(1/δ)/T}\right)$ for function value gaps in the convex setting, and for the squared gradient norms in the non-convex setting. Furthermore, when the noise parameters are sufficiently small, the convergence rate improves to $\tilde{O}\left(\log(1/δ)/T\right)$, where $T$ denotes the total number of iterations and $δ$ is the probability margin. Our analysis is also applicable to SGD with both constant and adaptive step sizes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11125 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations Yu, Chenhao Hong, Yusu Lin, Junhong Optimization and Control We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance assumptions for the stochastic gradient noise, we establish high-probability convergence rates of order $\tilde{O}\left(\sqrt{\log(1/δ)/T}\right)$ for function value gaps in the convex setting, and for the squared gradient norms in the non-convex setting. Furthermore, when the noise parameters are sufficiently small, the convergence rate improves to $\tilde{O}\left(\log(1/δ)/T\right)$, where $T$ denotes the total number of iterations and $δ$ is the probability margin. Our analysis is also applicable to SGD with both constant and adaptive step sizes. |
| title | Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2502.11125 |