Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908232681783296 |
|---|---|
| author | Ou, Zijing Zhang, Mingtian Zhang, Andi Xiao, Tim Z. Li, Yingzhen Barber, David |
| author_facet | Ou, Zijing Zhang, Mingtian Zhang, Andi Xiao, Tim Z. Li, Yingzhen Barber, David |
| contents | The probabilistic diffusion model has become highly effective across various domains. Typically, sampling from a diffusion model involves using a denoising distribution characterized by a Gaussian with a learned mean and either fixed or learned covariances. In this paper, we leverage the recently proposed covariance moment matching technique and introduce a novel method for learning the diagonal covariance. Unlike traditional data-driven diagonal covariance approximation approaches, our method involves directly regressing the optimal diagonal analytic covariance using a new, unbiased objective named Optimal Covariance Matching (OCM). This approach can significantly reduce the approximation error in covariance prediction. We demonstrate how our method can substantially enhance the sampling efficiency, recall rate and likelihood of commonly used diffusion models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10808 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching Ou, Zijing Zhang, Mingtian Zhang, Andi Xiao, Tim Z. Li, Yingzhen Barber, David Machine Learning The probabilistic diffusion model has become highly effective across various domains. Typically, sampling from a diffusion model involves using a denoising distribution characterized by a Gaussian with a learned mean and either fixed or learned covariances. In this paper, we leverage the recently proposed covariance moment matching technique and introduce a novel method for learning the diagonal covariance. Unlike traditional data-driven diagonal covariance approximation approaches, our method involves directly regressing the optimal diagonal analytic covariance using a new, unbiased objective named Optimal Covariance Matching (OCM). This approach can significantly reduce the approximation error in covariance prediction. We demonstrate how our method can substantially enhance the sampling efficiency, recall rate and likelihood of commonly used diffusion models. |
| title | Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.10808 |