A Framework for Variational Inference of Lightweight Bayesian Neural Networks with Heteroscedastic Uncertainties
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915971051028480 |
|---|---|
| author | Schodt, David J. Brown, Ryan Merritt, Michael Park, Samuel Menolascino, Delsin Peot, Mark A. |
| author_facet | Schodt, David J. Brown, Ryan Merritt, Michael Park, Samuel Menolascino, Delsin Peot, Mark A. |
| contents | Obtaining heteroscedastic predictive uncertainties from a Bayesian Neural Network (BNN) is vital to many applications. Often, heteroscedastic aleatoric uncertainties are learned as outputs of the BNN in addition to the predictive means, however doing so may necessitate adding more learnable parameters to the network. In this work, we demonstrate that both the heteroscedastic aleatoric and epistemic variance can be embedded into the variances of learned BNN parameters, improving predictive performance for lightweight networks. By complementing this approach with a moment propagation approach to inference, we introduce a relatively simple framework for sampling-free variational inference suitable for lightweight BNNs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14532 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Framework for Variational Inference of Lightweight Bayesian Neural Networks with Heteroscedastic Uncertainties Schodt, David J. Brown, Ryan Merritt, Michael Park, Samuel Menolascino, Delsin Peot, Mark A. Machine Learning Obtaining heteroscedastic predictive uncertainties from a Bayesian Neural Network (BNN) is vital to many applications. Often, heteroscedastic aleatoric uncertainties are learned as outputs of the BNN in addition to the predictive means, however doing so may necessitate adding more learnable parameters to the network. In this work, we demonstrate that both the heteroscedastic aleatoric and epistemic variance can be embedded into the variances of learned BNN parameters, improving predictive performance for lightweight networks. By complementing this approach with a moment propagation approach to inference, we introduce a relatively simple framework for sampling-free variational inference suitable for lightweight BNNs. |
| title | A Framework for Variational Inference of Lightweight Bayesian Neural Networks with Heteroscedastic Uncertainties |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.14532 |