Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908310189375488 |
|---|---|
| author | Zhang, Leyang Zhang, Yaoyu Luo, Tao |
| author_facet | Zhang, Leyang Zhang, Yaoyu Luo, Tao |
| contents | Under mild assumptions, we investigate the geometry of the loss landscape for two-layer neural networks in the vicinity of global minima. Utilizing novel techniques, we demonstrate: (i) how global minima with zero generalization error become geometrically separated from other global minima as the sample size grows; and (ii) the local convergence properties and rate of gradient flow dynamics. Our results indicate that two-layer neural networks can be locally recovered in the regime of overparameterization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_00508 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization Zhang, Leyang Zhang, Yaoyu Luo, Tao Machine Learning Dynamical Systems Under mild assumptions, we investigate the geometry of the loss landscape for two-layer neural networks in the vicinity of global minima. Utilizing novel techniques, we demonstrate: (i) how global minima with zero generalization error become geometrically separated from other global minima as the sample size grows; and (ii) the local convergence properties and rate of gradient flow dynamics. Our results indicate that two-layer neural networks can be locally recovered in the regime of overparameterization. |
| title | Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization |
| topic | Machine Learning Dynamical Systems |
| url | https://arxiv.org/abs/2309.00508 |