Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916627108331520 |
|---|---|
| author | Yu, Chenhao Hong, Yusu Lin, Junhong |
| author_facet | Yu, Chenhao Hong, Yusu Lin, Junhong |
| contents | We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance assumptions for the stochastic gradient noise, we establish high-probability convergence rates of order $\tilde{O}\left(\sqrt{\log(1/δ)/T}\right)$ for function value gaps in the convex setting, and for the squared gradient norms in the non-convex setting. Furthermore, when the noise parameters are sufficiently small, the convergence rate improves to $\tilde{O}\left(\log(1/δ)/T\right)$, where $T$ denotes the total number of iterations and $δ$ is the probability margin. Our analysis is also applicable to SGD with both constant and adaptive step sizes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11125 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations Yu, Chenhao Hong, Yusu Lin, Junhong Optimization and Control We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance assumptions for the stochastic gradient noise, we establish high-probability convergence rates of order $\tilde{O}\left(\sqrt{\log(1/δ)/T}\right)$ for function value gaps in the convex setting, and for the squared gradient norms in the non-convex setting. Furthermore, when the noise parameters are sufficiently small, the convergence rate improves to $\tilde{O}\left(\log(1/δ)/T\right)$, where $T$ denotes the total number of iterations and $δ$ is the probability margin. Our analysis is also applicable to SGD with both constant and adaptive step sizes. |
| title | Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2502.11125 |