Implicit Regularization of Large Neural Networks via Mean-Field Formulation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910062655569920 |
|---|---|
| author | Acciaio, Beatrice Heiss, Jakob Pammer, Gudmund Yan, Qinxin |
| author_facet | Acciaio, Beatrice Heiss, Jakob Pammer, Gudmund Yan, Qinxin |
| contents | We propose a mathematical framework to explain implicit regularization from early stopping during the training of overparametrized neural networks. In the mean-field limit, the parameter distribution evolves according to a gradient flow on the space of probability measures. We show that these dynamics admit an equivalent McKean-Vlasov stochastic control formulation through the corresponding Hamilton-Jacobi-Bellman (HJB) equation. The control viewpoint yields a Dynamic Programming Principle (DPP), which we use to define a new metric on probability measures. This metric can be viewed as a mean-field generalization of the control representation of the Wasserstein-2 distance, and it naturally appears as a regularization term selected by early stopping. We further obtain non-asymptotic bounds describing how the induced regularization depends on the stopping time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_20892 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Implicit Regularization of Large Neural Networks via Mean-Field Formulation Acciaio, Beatrice Heiss, Jakob Pammer, Gudmund Yan, Qinxin Optimization and Control Probability 49L20, 49L25, 49Q22, 68T07, 49N80 We propose a mathematical framework to explain implicit regularization from early stopping during the training of overparametrized neural networks. In the mean-field limit, the parameter distribution evolves according to a gradient flow on the space of probability measures. We show that these dynamics admit an equivalent McKean-Vlasov stochastic control formulation through the corresponding Hamilton-Jacobi-Bellman (HJB) equation. The control viewpoint yields a Dynamic Programming Principle (DPP), which we use to define a new metric on probability measures. This metric can be viewed as a mean-field generalization of the control representation of the Wasserstein-2 distance, and it naturally appears as a regularization term selected by early stopping. We further obtain non-asymptotic bounds describing how the induced regularization depends on the stopping time. |
| title | Implicit Regularization of Large Neural Networks via Mean-Field Formulation |
| topic | Optimization and Control Probability 49L20, 49L25, 49Q22, 68T07, 49N80 |
| url | https://arxiv.org/abs/2603.20892 |