Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2505.18535 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914366645862400 |
|---|---|
| author | Dudukalov, Dmitry Logachov, Artem Lotov, Vladimir Prasolov, Timofei Prokopenko, Evgeny Tarasenko, Anton |
| author_facet | Dudukalov, Dmitry Logachov, Artem Lotov, Vladimir Prasolov, Timofei Prokopenko, Evgeny Tarasenko, Anton |
| contents | We study the convergence properties and escape dynamics of Stochastic Gradient Descent (SGD) in one-dimensional landscapes, separately considering infinite- and finite-variance noise. Our main focus is to identify the time scales on which SGD reliably moves from an initial point to the local minimum in the same ''basin''. Under suitable conditions on the noise distribution, we prove that SGD converges to the basin's minimum unless the initial point lies too close to a local maximum. In that near-maximum scenario, we show that SGD can linger for a long time in its neighborhood. For initial points near a ''sharp'' maximum, we show that SGD does not remain stuck there, and we provide results to estimate the probability that it will reach each of the two neighboring minima. Overall, our findings present a nuanced view of SGD's transitions between local maxima and minima, influenced by both noise characteristics and the underlying function geometry. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18535 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD Dudukalov, Dmitry Logachov, Artem Lotov, Vladimir Prasolov, Timofei Prokopenko, Evgeny Tarasenko, Anton Machine Learning Probability We study the convergence properties and escape dynamics of Stochastic Gradient Descent (SGD) in one-dimensional landscapes, separately considering infinite- and finite-variance noise. Our main focus is to identify the time scales on which SGD reliably moves from an initial point to the local minimum in the same ''basin''. Under suitable conditions on the noise distribution, we prove that SGD converges to the basin's minimum unless the initial point lies too close to a local maximum. In that near-maximum scenario, we show that SGD can linger for a long time in its neighborhood. For initial points near a ''sharp'' maximum, we show that SGD does not remain stuck there, and we provide results to estimate the probability that it will reach each of the two neighboring minima. Overall, our findings present a nuanced view of SGD's transitions between local maxima and minima, influenced by both noise characteristics and the underlying function geometry. |
| title | Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD |
| topic | Machine Learning Probability |
| url | https://arxiv.org/abs/2505.18535 |