Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916203030642688 |
|---|---|
| author | Reichelt, Tim Ong, Luke Rainforth, Tom |
| author_facet | Reichelt, Tim Ong, Luke Rainforth, Tom |
| contents | The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path. We show that making predictions with this full posterior implicitly performs a Bayesian model averaging (BMA) over paths. This is potentially problematic, as BMA weights can be unstable due to model misspecification or inference approximations, leading to sub-optimal predictions in turn. To remedy this issue, we propose alternative mechanisms for path weighting: one based on stacking and one based on ideas from PAC-Bayes. We show how both can be implemented as a cheap post-processing step on top of existing inference engines. In our experiments, we find them to be more robust and lead to better predictions compared to the default BMA weights. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_14888 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support Reichelt, Tim Ong, Luke Rainforth, Tom Machine Learning Programming Languages The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path. We show that making predictions with this full posterior implicitly performs a Bayesian model averaging (BMA) over paths. This is potentially problematic, as BMA weights can be unstable due to model misspecification or inference approximations, leading to sub-optimal predictions in turn. To remedy this issue, we propose alternative mechanisms for path weighting: one based on stacking and one based on ideas from PAC-Bayes. We show how both can be implemented as a cheap post-processing step on top of existing inference engines. In our experiments, we find them to be more robust and lead to better predictions compared to the default BMA weights. |
| title | Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support |
| topic | Machine Learning Programming Languages |
| url | https://arxiv.org/abs/2310.14888 |