Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908881062461440 |
|---|---|
| author | Di Carlo, Luca Goddard, Chase Schwab, David J. |
| author_facet | Di Carlo, Luca Goddard, Chase Schwab, David J. |
| contents | Modern neural networks exhibit a striking property: basins of attraction in the loss landscape are often connected by low-loss paths, yet optimization dynamics generally remain confined to a single convex basin and rarely explore intermediate points. We resolve this paradox by identifying entropic barriers arising from the interplay between curvature variations along these paths and noise in optimization dynamics. Empirically, we find that curvature systematically rises away from minima, producing effective forces that bias noisy dynamics back toward the endpoints - even when the loss remains nearly flat. These barriers persist longer than energetic barriers, shaping the late-time localization of solutions in parameter space. Our results highlight the role of curvature-induced entropic forces in governing both connectivity and confinement in deep learning landscapes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_06297 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks Di Carlo, Luca Goddard, Chase Schwab, David J. Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence Modern neural networks exhibit a striking property: basins of attraction in the loss landscape are often connected by low-loss paths, yet optimization dynamics generally remain confined to a single convex basin and rarely explore intermediate points. We resolve this paradox by identifying entropic barriers arising from the interplay between curvature variations along these paths and noise in optimization dynamics. Empirically, we find that curvature systematically rises away from minima, producing effective forces that bias noisy dynamics back toward the endpoints - even when the loss remains nearly flat. These barriers persist longer than energetic barriers, shaping the late-time localization of solutions in parameter space. Our results highlight the role of curvature-induced entropic forces in governing both connectivity and confinement in deep learning landscapes. |
| title | Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks |
| topic | Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence |
| url | https://arxiv.org/abs/2512.06297 |