Paths and Ambient Spaces in Neural Loss Landscapes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917946449723392 |
|---|---|
| author | Dold, Daniel Kobialka, Julius Palm, Nicolai Sommer, Emanuel Rügamer, David Dürr, Oliver |
| author_facet | Dold, Daniel Kobialka, Julius Palm, Nicolai Sommer, Emanuel Rügamer, David Dürr, Oliver |
| contents | Understanding the structure of neural network loss surfaces, particularly the emergence of low-loss tunnels, is critical for advancing neural network theory and practice. In this paper, we propose a novel approach to directly embed loss tunnels into the loss landscape of neural networks. Exploring the properties of these loss tunnels offers new insights into their length and structure and sheds light on some common misconceptions. We then apply our approach to Bayesian neural networks, where we improve subspace inference by identifying pitfalls and proposing a more natural prior that better guides the sampling procedure. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_03382 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Paths and Ambient Spaces in Neural Loss Landscapes Dold, Daniel Kobialka, Julius Palm, Nicolai Sommer, Emanuel Rügamer, David Dürr, Oliver Machine Learning Understanding the structure of neural network loss surfaces, particularly the emergence of low-loss tunnels, is critical for advancing neural network theory and practice. In this paper, we propose a novel approach to directly embed loss tunnels into the loss landscape of neural networks. Exploring the properties of these loss tunnels offers new insights into their length and structure and sheds light on some common misconceptions. We then apply our approach to Bayesian neural networks, where we improve subspace inference by identifying pitfalls and proposing a more natural prior that better guides the sampling procedure. |
| title | Paths and Ambient Spaces in Neural Loss Landscapes |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.03382 |