Rethinking Diffusion Model in High Dimension
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909815777787904 |
|---|---|
| author | Zheng, Zhenxin Zheng, Zhenjie |
| author_facet | Zheng, Zhenxin Zheng, Zhenjie |
| contents | Curse of Dimensionality is an unavoidable challenge in statistical probability models, yet diffusion models seem to overcome this limitation, achieving impressive results in high-dimensional data generation. Diffusion models assume that they can learn the statistical quantities of the underlying probability distribution, enabling sampling from this distribution to generate realistic samples. But is this really how they work? We argue not, based on the following observations: 1) In high-dimensional sparse scenarios, the fitting target of the diffusion model's objective function degrades from a weighted sum of multiple samples to a single sample, which we believe hinders the model's ability to effectively learn essential statistical quantities such as posterior, score, or velocity field. 2) Most inference methods can be unified within a simple framework which involves no statistical concepts, aligns with the degraded objective function, and provides an novel and intuitive perspective on the inference process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_08643 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Diffusion Model in High Dimension Zheng, Zhenxin Zheng, Zhenjie Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Curse of Dimensionality is an unavoidable challenge in statistical probability models, yet diffusion models seem to overcome this limitation, achieving impressive results in high-dimensional data generation. Diffusion models assume that they can learn the statistical quantities of the underlying probability distribution, enabling sampling from this distribution to generate realistic samples. But is this really how they work? We argue not, based on the following observations: 1) In high-dimensional sparse scenarios, the fitting target of the diffusion model's objective function degrades from a weighted sum of multiple samples to a single sample, which we believe hinders the model's ability to effectively learn essential statistical quantities such as posterior, score, or velocity field. 2) Most inference methods can be unified within a simple framework which involves no statistical concepts, aligns with the degraded objective function, and provides an novel and intuitive perspective on the inference process. |
| title | Rethinking Diffusion Model in High Dimension |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.08643 |