On non-approximability of zero loss global ${\mathcal L}^2$ minimizers by gradient descent in Deep Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915299039641600 |
|---|---|
| author | Chen, Thomas Ewald, Patricia Muñoz |
| author_facet | Chen, Thomas Ewald, Patricia Muñoz |
| contents | We analyze geometric aspects of the gradient descent algorithm in Deep Learning (DL), and give a detailed discussion of the circumstance that in underparametrized DL networks, zero loss minimization can generically not be attained. As a consequence, we conclude that the distribution of training inputs must necessarily be non-generic in order to produce zero loss minimizers, both for the method constructed in [Chen-Munoz Ewald 2023, 2024], or for gradient descent [Chen 2025] (which assume clustering of training data). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_07065 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On non-approximability of zero loss global ${\mathcal L}^2$ minimizers by gradient descent in Deep Learning Chen, Thomas Ewald, Patricia Muñoz Machine Learning Artificial Intelligence Mathematical Physics Optimization and Control 57R70, 62M45 We analyze geometric aspects of the gradient descent algorithm in Deep Learning (DL), and give a detailed discussion of the circumstance that in underparametrized DL networks, zero loss minimization can generically not be attained. As a consequence, we conclude that the distribution of training inputs must necessarily be non-generic in order to produce zero loss minimizers, both for the method constructed in [Chen-Munoz Ewald 2023, 2024], or for gradient descent [Chen 2025] (which assume clustering of training data). |
| title | On non-approximability of zero loss global ${\mathcal L}^2$ minimizers by gradient descent in Deep Learning |
| topic | Machine Learning Artificial Intelligence Mathematical Physics Optimization and Control 57R70, 62M45 |
| url | https://arxiv.org/abs/2311.07065 |