Posterior SBC: Simulation-Based Calibration Checking Conditional on Data
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909532286877696 |
|---|---|
| author | Säilynoja, Teemu Schmitt, Marvin Bürkner, Paul-Christian Vehtari, Aki |
| author_facet | Säilynoja, Teemu Schmitt, Marvin Bürkner, Paul-Christian Vehtari, Aki |
| contents | Simulation-based calibration checking (SBC) refers to the validation of an inference algorithm and model implementation through repeated inference on data simulated from a generative model. In the original and commonly used approach, the generative model uses parameters drawn from the prior, and thus the approach is testing whether the inference works for simulated data generated with parameter values plausible under that prior. This approach is natural and desirable when we want to test whether the inference works for a wide range of datasets we might observe. However, after observing data, we are interested in answering whether the inference works conditional on that particular data. In this paper, we propose posterior SBC and demonstrate how it can be used to validate the inference conditionally on observed data. We illustrate the utility of posterior SBC in three case studies: (1) A simple multilevel model; (2) a model that is governed by differential equations; and (3) a joint integrative neuroscience model which is approximated via amortized Bayesian inference with neural networks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_03279 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Posterior SBC: Simulation-Based Calibration Checking Conditional on Data Säilynoja, Teemu Schmitt, Marvin Bürkner, Paul-Christian Vehtari, Aki Methodology Computation Machine Learning Simulation-based calibration checking (SBC) refers to the validation of an inference algorithm and model implementation through repeated inference on data simulated from a generative model. In the original and commonly used approach, the generative model uses parameters drawn from the prior, and thus the approach is testing whether the inference works for simulated data generated with parameter values plausible under that prior. This approach is natural and desirable when we want to test whether the inference works for a wide range of datasets we might observe. However, after observing data, we are interested in answering whether the inference works conditional on that particular data. In this paper, we propose posterior SBC and demonstrate how it can be used to validate the inference conditionally on observed data. We illustrate the utility of posterior SBC in three case studies: (1) A simple multilevel model; (2) a model that is governed by differential equations; and (3) a joint integrative neuroscience model which is approximated via amortized Bayesian inference with neural networks. |
| title | Posterior SBC: Simulation-Based Calibration Checking Conditional on Data |
| topic | Methodology Computation Machine Learning |
| url | https://arxiv.org/abs/2502.03279 |