VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918131990003712 |
|---|---|
| author | Wang, Leyang Zhang, Mingtian Ou, Zijing Barber, David |
| author_facet | Wang, Leyang Zhang, Mingtian Ou, Zijing Barber, David |
| contents | Recently, diffusion distillation methods have compressed thousand-step teacher diffusion models into one-step student generators while preserving sample quality. Most existing approaches train the student model using a diffusive divergence whose gradient is approximated via the student's score function, learned through denoising score matching (DSM). Since DSM training is imperfect, the resulting gradient estimate is inevitably biased, leading to sub-optimal performance. In this paper, we propose VarDiU (pronounced /va:rdju:/), a Variational Diffusive Upper Bound that admits an unbiased gradient estimator and can be directly applied to diffusion distillation. Using this objective, we compare our method with Diff-Instruct and demonstrate that it achieves higher generation quality and enables a more efficient and stable training procedure for one-step diffusion distillation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20646 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation Wang, Leyang Zhang, Mingtian Ou, Zijing Barber, David Machine Learning Recently, diffusion distillation methods have compressed thousand-step teacher diffusion models into one-step student generators while preserving sample quality. Most existing approaches train the student model using a diffusive divergence whose gradient is approximated via the student's score function, learned through denoising score matching (DSM). Since DSM training is imperfect, the resulting gradient estimate is inevitably biased, leading to sub-optimal performance. In this paper, we propose VarDiU (pronounced /va:rdju:/), a Variational Diffusive Upper Bound that admits an unbiased gradient estimator and can be directly applied to diffusion distillation. Using this objective, we compare our method with Diff-Instruct and demonstrate that it achieves higher generation quality and enables a more efficient and stable training procedure for one-step diffusion distillation. |
| title | VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2508.20646 |