Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916262522650624 |
|---|---|
| author | Sommer, Emanuel Wimmer, Lisa Papamarkou, Theodore Bothmann, Ludwig Bischl, Bernd Rügamer, David |
| author_facet | Sommer, Emanuel Wimmer, Lisa Papamarkou, Theodore Bothmann, Ludwig Bischl, Bernd Rügamer, David |
| contents | A major challenge in sample-based inference (SBI) for Bayesian neural networks is the size and structure of the networks' parameter space. Our work shows that successful SBI is possible by embracing the characteristic relationship between weight and function space, uncovering a systematic link between overparameterization and the difficulty of the sampling problem. Through extensive experiments, we establish practical guidelines for sampling and convergence diagnosis. As a result, we present a deep ensemble initialized approach as an effective solution with competitive performance and uncertainty quantification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01484 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks? Sommer, Emanuel Wimmer, Lisa Papamarkou, Theodore Bothmann, Ludwig Bischl, Bernd Rügamer, David Machine Learning Computation A major challenge in sample-based inference (SBI) for Bayesian neural networks is the size and structure of the networks' parameter space. Our work shows that successful SBI is possible by embracing the characteristic relationship between weight and function space, uncovering a systematic link between overparameterization and the difficulty of the sampling problem. Through extensive experiments, we establish practical guidelines for sampling and convergence diagnosis. As a result, we present a deep ensemble initialized approach as an effective solution with competitive performance and uncertainty quantification. |
| title | Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks? |
| topic | Machine Learning Computation |
| url | https://arxiv.org/abs/2402.01484 |