Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912288310558720 |
|---|---|
| author | Rissanen, Severi Heinonen, Markus Solin, Arno |
| author_facet | Rissanen, Severi Heinonen, Markus Solin, Arno |
| contents | The covariance for clean data given a noisy observation is an important quantity in many training-free guided generation methods for diffusion models. Current methods require heavy test-time computation, altering the standard diffusion training process or denoiser architecture, or making heavy approximations. We propose a new framework that sidesteps these issues by using covariance information that is available for free from training data and the curvature of the generative trajectory, which is linked to the covariance through the second-order Tweedie's formula. We integrate these sources of information using (i) a novel method to transfer covariance estimates across noise levels and (ii) low-rank updates in a given noise level. We validate the method on linear inverse problems, where it outperforms recent baselines, especially with fewer diffusion steps. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_11149 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs Rissanen, Severi Heinonen, Markus Solin, Arno Machine Learning The covariance for clean data given a noisy observation is an important quantity in many training-free guided generation methods for diffusion models. Current methods require heavy test-time computation, altering the standard diffusion training process or denoiser architecture, or making heavy approximations. We propose a new framework that sidesteps these issues by using covariance information that is available for free from training data and the curvature of the generative trajectory, which is linked to the covariance through the second-order Tweedie's formula. We integrate these sources of information using (i) a novel method to transfer covariance estimates across noise levels and (ii) low-rank updates in a given noise level. We validate the method on linear inverse problems, where it outperforms recent baselines, especially with fewer diffusion steps. |
| title | Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.11149 |