Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2605.15459 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916015365947392 |
|---|---|
| author | Jacobsen, Albert Kjøller Jakobsen, Leo Uhre Gegenfurtner, Johanna Marie Arvanitidis, Georgios |
| author_facet | Jacobsen, Albert Kjøller Jakobsen, Leo Uhre Gegenfurtner, Johanna Marie Arvanitidis, Georgios |
| contents | The minima of modern neural network loss functions are typically not isolated, rather they form connected components of reparameterization invariant solutions on the training data. Analytically characterizing these solutions is a hard problem, but sampling approaches are feasible. By construction, existing methods either spread over low-loss regions, and thus do not sample reparameterization invariant solutions exactly, or are inherently local, which limits exploration of other minima valleys. We propose sampling such reparameterization invariant models using a dynamical system based on kinetic energy, subject to a gravitational pull and a friction term that dissipates energy from the system. Our proposed sampler, DiMS, is guaranteed to sample exactly from the minimum level sets and depends on physically motivated hyperparameters which allows control over the exploration capabilities of the sampler. We consider uncertainty quantification in Bayesian inference as the motivating problem and observe improved performance compared to previously proposed approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15459 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Don't Stop Me Yet: Sampling Loss Minima via Dissipative Riemannian Mechanics Jacobsen, Albert Kjøller Jakobsen, Leo Uhre Gegenfurtner, Johanna Marie Arvanitidis, Georgios Machine Learning The minima of modern neural network loss functions are typically not isolated, rather they form connected components of reparameterization invariant solutions on the training data. Analytically characterizing these solutions is a hard problem, but sampling approaches are feasible. By construction, existing methods either spread over low-loss regions, and thus do not sample reparameterization invariant solutions exactly, or are inherently local, which limits exploration of other minima valleys. We propose sampling such reparameterization invariant models using a dynamical system based on kinetic energy, subject to a gravitational pull and a friction term that dissipates energy from the system. Our proposed sampler, DiMS, is guaranteed to sample exactly from the minimum level sets and depends on physically motivated hyperparameters which allows control over the exploration capabilities of the sampler. We consider uncertainty quantification in Bayesian inference as the motivating problem and observe improved performance compared to previously proposed approaches. |
| title | Don't Stop Me Yet: Sampling Loss Minima via Dissipative Riemannian Mechanics |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.15459 |