The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910008038391808 |
|---|---|
| author | Erba, Vittorio Troiani, Emanuele Zdeborová, Lenka Krzakala, Florent |
| author_facet | Erba, Vittorio Troiani, Emanuele Zdeborová, Lenka Krzakala, Florent |
| contents | We study the high-dimensional asymptotics of empirical risk minimization (ERM) in over-parametrized two-layer neural networks with quadratic activations trained on synthetic data. We derive sharp asymptotics for both training and test errors by mapping the $\ell_2$-regularized learning problem to a convex matrix sensing task with nuclear norm penalization. This reveals that capacity control in such networks emerges from a low-rank structure in the learned feature maps. Our results characterize the global minima of the loss and yield precise generalization thresholds, showing how the width of the target function governs learnability. This analysis bridges and extends ideas from spin-glass methods, matrix factorization, and convex optimization and emphasizes the deep link between low-rank matrix sensing and learning in quadratic neural networks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17958 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks Erba, Vittorio Troiani, Emanuele Zdeborová, Lenka Krzakala, Florent Machine Learning Disordered Systems and Neural Networks Information Theory We study the high-dimensional asymptotics of empirical risk minimization (ERM) in over-parametrized two-layer neural networks with quadratic activations trained on synthetic data. We derive sharp asymptotics for both training and test errors by mapping the $\ell_2$-regularized learning problem to a convex matrix sensing task with nuclear norm penalization. This reveals that capacity control in such networks emerges from a low-rank structure in the learned feature maps. Our results characterize the global minima of the loss and yield precise generalization thresholds, showing how the width of the target function governs learnability. This analysis bridges and extends ideas from spin-glass methods, matrix factorization, and convex optimization and emphasizes the deep link between low-rank matrix sensing and learning in quadratic neural networks. |
| title | The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks |
| topic | Machine Learning Disordered Systems and Neural Networks Information Theory |
| url | https://arxiv.org/abs/2505.17958 |