Incorporating Unlabelled Data into Bayesian Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914929196400640 |
|---|---|
| author | Sharma, Mrinank Rainforth, Tom Teh, Yee Whye Fortuin, Vincent |
| author_facet | Sharma, Mrinank Rainforth, Tom Teh, Yee Whye Fortuin, Vincent |
| contents | Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions. To overcome this limitation, we introduce Self-Supervised Bayesian Neural Networks, which use unlabelled data to learn models with suitable prior predictive distributions. This is achieved by leveraging contrastive pretraining techniques and optimising a variational lower bound. We then show that the prior predictive distributions of self-supervised BNNs capture problem semantics better than conventional BNN priors. In turn, our approach offers improved predictive performance over conventional BNNs, especially in low-budget regimes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_01762 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Incorporating Unlabelled Data into Bayesian Neural Networks Sharma, Mrinank Rainforth, Tom Teh, Yee Whye Fortuin, Vincent Machine Learning Artificial Intelligence Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions. To overcome this limitation, we introduce Self-Supervised Bayesian Neural Networks, which use unlabelled data to learn models with suitable prior predictive distributions. This is achieved by leveraging contrastive pretraining techniques and optimising a variational lower bound. We then show that the prior predictive distributions of self-supervised BNNs capture problem semantics better than conventional BNN priors. In turn, our approach offers improved predictive performance over conventional BNNs, especially in low-budget regimes. |
| title | Incorporating Unlabelled Data into Bayesian Neural Networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2304.01762 |