Gradient descent in matrix factorization: Understanding large initialization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910466271346688 |
|---|---|
| author | Chen, Hengchao Chen, Xin Elmasri, Mohamad Sun, Qiang |
| author_facet | Chen, Hengchao Chen, Xin Elmasri, Mohamad Sun, Qiang |
| contents | Gradient Descent (GD) has been proven effective in solving various matrix factorization problems. However, its optimization behavior with large initial values remains less understood. To address this gap, this paper presents a novel theoretical framework for examining the convergence trajectory of GD with a large initialization. The framework is grounded in signal-to-noise ratio concepts and inductive arguments. The results uncover an implicit incremental learning phenomenon in GD and offer a deeper understanding of its performance in large initialization scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_19206 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Gradient descent in matrix factorization: Understanding large initialization Chen, Hengchao Chen, Xin Elmasri, Mohamad Sun, Qiang Optimization and Control Machine Learning Gradient Descent (GD) has been proven effective in solving various matrix factorization problems. However, its optimization behavior with large initial values remains less understood. To address this gap, this paper presents a novel theoretical framework for examining the convergence trajectory of GD with a large initialization. The framework is grounded in signal-to-noise ratio concepts and inductive arguments. The results uncover an implicit incremental learning phenomenon in GD and offer a deeper understanding of its performance in large initialization scenarios. |
| title | Gradient descent in matrix factorization: Understanding large initialization |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2305.19206 |