Randomized Kaczmarz with geometrically smoothed momentum
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912000457572352 |
|---|---|
| author | Alderman, Seth J. Luikart, Roan W. Marshall, Nicholas F. |
| author_facet | Alderman, Seth J. Luikart, Roan W. Marshall, Nicholas F. |
| contents | This paper studies the effect of adding geometrically smoothed momentum to the randomized Kaczmarz algorithm, which is an instance of stochastic gradient descent on a linear least squares loss function. We prove a result about the expected error in the direction of singular vectors of the matrix defining the least squares loss. We present several numerical examples illustrating the utility of our result and pose several questions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_09415 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Randomized Kaczmarz with geometrically smoothed momentum Alderman, Seth J. Luikart, Roan W. Marshall, Nicholas F. Numerical Analysis Probability Machine Learning 65F10, 37N30, 60H25, 65F20 This paper studies the effect of adding geometrically smoothed momentum to the randomized Kaczmarz algorithm, which is an instance of stochastic gradient descent on a linear least squares loss function. We prove a result about the expected error in the direction of singular vectors of the matrix defining the least squares loss. We present several numerical examples illustrating the utility of our result and pose several questions. |
| title | Randomized Kaczmarz with geometrically smoothed momentum |
| topic | Numerical Analysis Probability Machine Learning 65F10, 37N30, 60H25, 65F20 |
| url | https://arxiv.org/abs/2401.09415 |