Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.07998 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910522763378688 |
|---|---|
| author | Singhal, Raghav Goldstein, Mark Ranganath, Rajesh |
| author_facet | Singhal, Raghav Goldstein, Mark Ranganath, Rajesh |
| contents | Reversing a diffusion process by learning its score forms the heart of diffusion-based generative modeling and for estimating properties of scientific systems. The diffusion processes that are tractable center on linear processes with a Gaussian stationary distribution. This limits the kinds of models that can be built to those that target a Gaussian prior or more generally limits the kinds of problems that can be generically solved to those that have conditionally linear score functions. In this work, we introduce a family of tractable denoising score matching objectives, called local-DSM, built using local increments of the diffusion process. We show how local-DSM melded with Taylor expansions enables automated training and score estimation with nonlinear diffusion processes. To demonstrate these ideas, we use automated-DSM to train generative models using non-Gaussian priors on challenging low dimensional distributions and the CIFAR10 image dataset. Additionally, we use the automated-DSM to learn the scores for nonlinear processes studied in statistical physics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_07998 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | What's the score? Automated Denoising Score Matching for Nonlinear Diffusions Singhal, Raghav Goldstein, Mark Ranganath, Rajesh Machine Learning Reversing a diffusion process by learning its score forms the heart of diffusion-based generative modeling and for estimating properties of scientific systems. The diffusion processes that are tractable center on linear processes with a Gaussian stationary distribution. This limits the kinds of models that can be built to those that target a Gaussian prior or more generally limits the kinds of problems that can be generically solved to those that have conditionally linear score functions. In this work, we introduce a family of tractable denoising score matching objectives, called local-DSM, built using local increments of the diffusion process. We show how local-DSM melded with Taylor expansions enables automated training and score estimation with nonlinear diffusion processes. To demonstrate these ideas, we use automated-DSM to train generative models using non-Gaussian priors on challenging low dimensional distributions and the CIFAR10 image dataset. Additionally, we use the automated-DSM to learn the scores for nonlinear processes studied in statistical physics. |
| title | What's the score? Automated Denoising Score Matching for Nonlinear Diffusions |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.07998 |