Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911509836201984 |
|---|---|
| author | Piriyakulkij, Wasu Top Wang, Yingheng Kuleshov, Volodymyr |
| author_facet | Piriyakulkij, Wasu Top Wang, Yingheng Kuleshov, Volodymyr |
| contents | We propose denoising diffusion variational inference (DDVI), a black-box variational inference algorithm for latent variable models which relies on diffusion models as flexible approximate posteriors. Specifically, our method introduces an expressive class of diffusion-based variational posteriors that perform iterative refinement in latent space; we train these posteriors with a novel regularized evidence lower bound (ELBO) on the marginal likelihood inspired by the wake-sleep algorithm. Our method is easy to implement (it fits a regularized extension of the ELBO), is compatible with black-box variational inference, and outperforms alternative classes of approximate posteriors based on normalizing flows or adversarial networks. We find that DDVI improves inference and learning in deep latent variable models across common benchmarks as well as on a motivating task in biology -- inferring latent ancestry from human genomes -- where it outperforms strong baselines on the Thousand Genomes dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_02739 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors Piriyakulkij, Wasu Top Wang, Yingheng Kuleshov, Volodymyr Machine Learning Quantitative Methods We propose denoising diffusion variational inference (DDVI), a black-box variational inference algorithm for latent variable models which relies on diffusion models as flexible approximate posteriors. Specifically, our method introduces an expressive class of diffusion-based variational posteriors that perform iterative refinement in latent space; we train these posteriors with a novel regularized evidence lower bound (ELBO) on the marginal likelihood inspired by the wake-sleep algorithm. Our method is easy to implement (it fits a regularized extension of the ELBO), is compatible with black-box variational inference, and outperforms alternative classes of approximate posteriors based on normalizing flows or adversarial networks. We find that DDVI improves inference and learning in deep latent variable models across common benchmarks as well as on a motivating task in biology -- inferring latent ancestry from human genomes -- where it outperforms strong baselines on the Thousand Genomes dataset. |
| title | Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors |
| topic | Machine Learning Quantitative Methods |
| url | https://arxiv.org/abs/2401.02739 |