On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910065986895872 |
|---|---|
| author | Kobialka, Julius Sommer, Emanuel Kolb, Chris Kwon, Juntae Dold, Daniel Rügamer, David |
| author_facet | Kobialka, Julius Sommer, Emanuel Kolb, Chris Kwon, Juntae Dold, Daniel Rügamer, David |
| contents | Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: balancedness, weight reallocation on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_22030 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors Kobialka, Julius Sommer, Emanuel Kolb, Chris Kwon, Juntae Dold, Daniel Rügamer, David Machine Learning Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: balancedness, weight reallocation on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions. |
| title | On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2603.22030 |