Improved Learning Rates for Stochastic Optimization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2021
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915872972472320 |
|---|---|
| author | Li, Shaojie Tang, Pengwei Liu, Yong |
| author_facet | Li, Shaojie Tang, Pengwei Liu, Yong |
| contents | Stochastic optimization is a cornerstone of modern machine learning. This paper studies the generalization performance of two classical stochastic optimization algorithms: stochastic gradient descent (SGD) and Nesterov's accelerated gradient (NAG). We establish new learning rates for both algorithms, with improved guarantees in some settings or comparable rates under weaker assumptions in others. We also provide numerical experiments to support the theory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2107_08686 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Improved Learning Rates for Stochastic Optimization Li, Shaojie Tang, Pengwei Liu, Yong Machine Learning Optimization and Control Stochastic optimization is a cornerstone of modern machine learning. This paper studies the generalization performance of two classical stochastic optimization algorithms: stochastic gradient descent (SGD) and Nesterov's accelerated gradient (NAG). We establish new learning rates for both algorithms, with improved guarantees in some settings or comparable rates under weaker assumptions in others. We also provide numerical experiments to support the theory. |
| title | Improved Learning Rates for Stochastic Optimization |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2107.08686 |