On the SAGA algorithm with decreasing step
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916424446902272 |
|---|---|
| author | Fredes, Luis Bercu, Bernard Gbaguidi, Eméric |
| author_facet | Fredes, Luis Bercu, Bernard Gbaguidi, Eméric |
| contents | Stochastic optimization naturally appear in many application areas, including machine learning. Our goal is to go further in the analysis of the Stochastic Average Gradient Accelerated (SAGA) algorithm. To achieve this, we introduce a new $λ$-SAGA algorithm which interpolates between the Stochastic Gradient Descent ($λ=0$) and the SAGA algorithm ($λ=1$). Firstly, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated to the objective function. Secondly, we establish a central limit theorem for the $λ$-SAGA algorithm. Finally, we provide the non-asymptotic $\mathbb{L}^p$ rates of convergence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03760 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the SAGA algorithm with decreasing step Fredes, Luis Bercu, Bernard Gbaguidi, Eméric Optimization and Control Machine Learning Probability Stochastic optimization naturally appear in many application areas, including machine learning. Our goal is to go further in the analysis of the Stochastic Average Gradient Accelerated (SAGA) algorithm. To achieve this, we introduce a new $λ$-SAGA algorithm which interpolates between the Stochastic Gradient Descent ($λ=0$) and the SAGA algorithm ($λ=1$). Firstly, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated to the objective function. Secondly, we establish a central limit theorem for the $λ$-SAGA algorithm. Finally, we provide the non-asymptotic $\mathbb{L}^p$ rates of convergence. |
| title | On the SAGA algorithm with decreasing step |
| topic | Optimization and Control Machine Learning Probability |
| url | https://arxiv.org/abs/2410.03760 |