Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929208023842816 |
|---|---|
| author | An, Jing Lu, Jianfeng |
| author_facet | An, Jing Lu, Jianfeng |
| contents | We study the convergence of stochastic gradient descent (SGD) for non-convex objective functions. We establish the local convergence with positive probability under the local Łojasiewicz condition introduced by Chatterjee in \cite{chatterjee2022convergence} and an additional local structural assumption of the loss function landscape. A key component of our proof is to ensure that the whole trajectories of SGD stay inside the local region with a positive probability. We also provide examples of neural networks with finite widths such that our assumptions hold. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_09221 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks An, Jing Lu, Jianfeng Machine Learning Optimization and Control We study the convergence of stochastic gradient descent (SGD) for non-convex objective functions. We establish the local convergence with positive probability under the local Łojasiewicz condition introduced by Chatterjee in \cite{chatterjee2022convergence} and an additional local structural assumption of the loss function landscape. A key component of our proof is to ensure that the whole trajectories of SGD stay inside the local region with a positive probability. We also provide examples of neural networks with finite widths such that our assumptions hold. |
| title | Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2304.09221 |