Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.12786 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916619377180672 |
|---|---|
| author | Thornton, James Bethune, Louis Zhang, Ruixiang Bradley, Arwen Nakkiran, Preetum Zhai, Shuangfei |
| author_facet | Thornton, James Bethune, Louis Zhang, Ruixiang Bradley, Arwen Nakkiran, Preetum Zhai, Shuangfei |
| contents | Diffusion models may be formulated as a time-indexed sequence of energy-based models, where the score corresponds to the negative gradient of an energy function. As opposed to learning the score directly, an energy parameterization is attractive as the energy itself can be used to control generation via Monte Carlo samplers. Architectural constraints and training instability in energy parameterized models have so far yielded inferior performance compared to directly approximating the score or denoiser. We address these deficiencies by introducing a novel training regime for the energy function through distillation of pre-trained diffusion models, resembling a Helmholtz decomposition of the score vector field. We further showcase the synergies between energy and score by casting the diffusion sampling procedure as a Feynman Kac model where sampling is controlled using potentials from the learnt energy functions. The Feynman Kac model formalism enables composition and low temperature sampling through sequential Monte Carlo. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12786 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo Thornton, James Bethune, Louis Zhang, Ruixiang Bradley, Arwen Nakkiran, Preetum Zhai, Shuangfei Machine Learning Diffusion models may be formulated as a time-indexed sequence of energy-based models, where the score corresponds to the negative gradient of an energy function. As opposed to learning the score directly, an energy parameterization is attractive as the energy itself can be used to control generation via Monte Carlo samplers. Architectural constraints and training instability in energy parameterized models have so far yielded inferior performance compared to directly approximating the score or denoiser. We address these deficiencies by introducing a novel training regime for the energy function through distillation of pre-trained diffusion models, resembling a Helmholtz decomposition of the score vector field. We further showcase the synergies between energy and score by casting the diffusion sampling procedure as a Feynman Kac model where sampling is controlled using potentials from the learnt energy functions. The Feynman Kac model formalism enables composition and low temperature sampling through sequential Monte Carlo. |
| title | Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.12786 |