Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2307.01050 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918012367405056 |
|---|---|
| author | Vargas, Francisco Padhy, Shreyas Blessing, Denis Nüsken, Nikolas |
| author_facet | Vargas, Francisco Padhy, Shreyas Blessing, Denis Nüsken, Nikolas |
| contents | Connecting optimal transport and variational inference, we present a principled and systematic framework for sampling and generative modelling centred around divergences on path space. Our work culminates in the development of the \emph{Controlled Monte Carlo Diffusion} sampler (CMCD) for Bayesian computation, a score-based annealing technique that crucially adapts both forward and backward dynamics in a diffusion model. On the way, we clarify the relationship between the EM-algorithm and iterative proportional fitting (IPF) for Schr{ö}dinger bridges, deriving as well a regularised objective that bypasses the iterative bottleneck of standard IPF-updates. Finally, we show that CMCD has a strong foundation in the Jarzinsky and Crooks identities from statistical physics, and that it convincingly outperforms competing approaches across a wide array of experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_01050 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Transport meets Variational Inference: Controlled Monte Carlo Diffusions Vargas, Francisco Padhy, Shreyas Blessing, Denis Nüsken, Nikolas Machine Learning Connecting optimal transport and variational inference, we present a principled and systematic framework for sampling and generative modelling centred around divergences on path space. Our work culminates in the development of the \emph{Controlled Monte Carlo Diffusion} sampler (CMCD) for Bayesian computation, a score-based annealing technique that crucially adapts both forward and backward dynamics in a diffusion model. On the way, we clarify the relationship between the EM-algorithm and iterative proportional fitting (IPF) for Schr{ö}dinger bridges, deriving as well a regularised objective that bypasses the iterative bottleneck of standard IPF-updates. Finally, we show that CMCD has a strong foundation in the Jarzinsky and Crooks identities from statistical physics, and that it convincingly outperforms competing approaches across a wide array of experiments. |
| title | Transport meets Variational Inference: Controlled Monte Carlo Diffusions |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2307.01050 |