Stochastic Adaptive Gradient Descent Without Descent
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915501331972096 |
|---|---|
| author | Aujol, Jean-François Bigot, Jérémie Castera, Camille |
| author_facet | Aujol, Jean-François Bigot, Jérémie Castera, Camille |
| contents | We introduce a new adaptive step-size strategy for convex optimization with stochastic gradient that exploits the local geometry of the objective function only by means of a first-order stochastic oracle and without any hyper-parameter tuning. The method comes from a theoretically-grounded adaptation of the Adaptive Gradient Descent Without Descent method to the stochastic setting. We prove the convergence of stochastic gradient descent with our step-size under various assumptions, and we show that it empirically competes against tuned baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14969 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Stochastic Adaptive Gradient Descent Without Descent Aujol, Jean-François Bigot, Jérémie Castera, Camille Machine Learning Optimization and Control We introduce a new adaptive step-size strategy for convex optimization with stochastic gradient that exploits the local geometry of the objective function only by means of a first-order stochastic oracle and without any hyper-parameter tuning. The method comes from a theoretically-grounded adaptation of the Adaptive Gradient Descent Without Descent method to the stochastic setting. We prove the convergence of stochastic gradient descent with our step-size under various assumptions, and we show that it empirically competes against tuned baselines. |
| title | Stochastic Adaptive Gradient Descent Without Descent |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2509.14969 |