Minimaxity and Admissibility of Bayesian Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913007662006272 |
|---|---|
| author | Coulson, Daniel Andrew Wells, Martin T. |
| author_facet | Coulson, Daniel Andrew Wells, Martin T. |
| contents | Bayesian neural networks (BNNs) offer a natural probabilistic formulation for inference in deep learning models. Despite their popularity, their optimality has received limited attention through the lens of statistical decision theory. In this paper, we study decision rules induced by deep, fully connected feedforward ReLU BNNs in the normal location model under quadratic loss. We show that, for fixed prior scales, the induced Bayes decision rule is not minimax. We then propose a hyperprior on the effective output variance of the BNN prior that yields a superharmonic square-root marginal density, establishing that the resulting decision rule is simultaneously admissible and minimax. We further extend these results from the quadratic loss setting to the predictive density estimation problem with Kullback--Leibler loss. Finally, we validate our theoretical findings numerically through simulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_04673 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Minimaxity and Admissibility of Bayesian Neural Networks Coulson, Daniel Andrew Wells, Martin T. Statistics Theory Machine Learning Primary 62C10, 62C15, 62C20, 62M45, secondary 62A15, 62F15 Bayesian neural networks (BNNs) offer a natural probabilistic formulation for inference in deep learning models. Despite their popularity, their optimality has received limited attention through the lens of statistical decision theory. In this paper, we study decision rules induced by deep, fully connected feedforward ReLU BNNs in the normal location model under quadratic loss. We show that, for fixed prior scales, the induced Bayes decision rule is not minimax. We then propose a hyperprior on the effective output variance of the BNN prior that yields a superharmonic square-root marginal density, establishing that the resulting decision rule is simultaneously admissible and minimax. We further extend these results from the quadratic loss setting to the predictive density estimation problem with Kullback--Leibler loss. Finally, we validate our theoretical findings numerically through simulation. |
| title | Minimaxity and Admissibility of Bayesian Neural Networks |
| topic | Statistics Theory Machine Learning Primary 62C10, 62C15, 62C20, 62M45, secondary 62A15, 62F15 |
| url | https://arxiv.org/abs/2604.04673 |