Iterative Importance Fine-tuning of Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910018204336128 |
|---|---|
| author | Denker, Alexander Padhy, Shreyas Vargas, Francisco Hertrich, Johannes |
| author_facet | Denker, Alexander Padhy, Shreyas Vargas, Francisco Hertrich, Johannes |
| contents | Diffusion models are an important tool for generative modelling, serving as effective priors in applications such as imaging and protein design. A key challenge in applying diffusion models for downstream tasks is efficiently sampling from resulting posterior distributions, which can be addressed using Doob's $h$-transform. This work introduces a self-supervised algorithm for fine-tuning diffusion models by learning the optimal control, enabling amortised conditional sampling. Our method iteratively refines the control using a synthetic dataset resampled with path-based importance weights. We demonstrate the effectiveness of this framework on class-conditional sampling, inverse problems and reward fine-tuning for text-to-image diffusion models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_04468 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Iterative Importance Fine-tuning of Diffusion Models Denker, Alexander Padhy, Shreyas Vargas, Francisco Hertrich, Johannes Machine Learning Image and Video Processing Probability 68T07 I.4.9; I.2.6 Diffusion models are an important tool for generative modelling, serving as effective priors in applications such as imaging and protein design. A key challenge in applying diffusion models for downstream tasks is efficiently sampling from resulting posterior distributions, which can be addressed using Doob's $h$-transform. This work introduces a self-supervised algorithm for fine-tuning diffusion models by learning the optimal control, enabling amortised conditional sampling. Our method iteratively refines the control using a synthetic dataset resampled with path-based importance weights. We demonstrate the effectiveness of this framework on class-conditional sampling, inverse problems and reward fine-tuning for text-to-image diffusion models. |
| title | Iterative Importance Fine-tuning of Diffusion Models |
| topic | Machine Learning Image and Video Processing Probability 68T07 I.4.9; I.2.6 |
| url | https://arxiv.org/abs/2502.04468 |