Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910538399744000 |
|---|---|
| author | Schmitt, Marvin Ivanova, Desi R. Habermann, Daniel Köthe, Ullrich Bürkner, Paul-Christian Radev, Stefan T. |
| author_facet | Schmitt, Marvin Ivanova, Desi R. Habermann, Daniel Köthe, Ullrich Bürkner, Paul-Christian Radev, Stefan T. |
| contents | We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a \textit{self-consistency loss} which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_04395 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference Schmitt, Marvin Ivanova, Desi R. Habermann, Daniel Köthe, Ullrich Bürkner, Paul-Christian Radev, Stefan T. Machine Learning Artificial Intelligence We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a \textit{self-consistency loss} which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation. |
| title | Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.04395 |