Analysis of Diffusion Models for Manifold Data
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917079740841984 |
|---|---|
| author | George, Anand Jerry Veiga, Rodrigo Macris, Nicolas |
| author_facet | George, Anand Jerry Veiga, Rodrigo Macris, Nicolas |
| contents | We analyze the time reversed dynamics of generative diffusion models. If the exact empirical score function is used in a regime of large dimension and exponentially large number of samples, these models are known to undergo transitions between distinct dynamical regimes. We extend this analysis and compute the transitions for an analytically tractable manifold model where the statistical model for the data is a mixture of lower dimensional Gaussians embedded in higher dimensional space. We compute the so-called speciation and collapse transition times, as a function of the ratio of manifold-to-ambient space dimensions, and other characteristics of the data model. An important tool used in our analysis is the exact formula for the mutual information (or free energy) of Generalized Linear Models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_04339 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Analysis of Diffusion Models for Manifold Data George, Anand Jerry Veiga, Rodrigo Macris, Nicolas Statistics Theory Disordered Systems and Neural Networks Information Theory Machine Learning Probability We analyze the time reversed dynamics of generative diffusion models. If the exact empirical score function is used in a regime of large dimension and exponentially large number of samples, these models are known to undergo transitions between distinct dynamical regimes. We extend this analysis and compute the transitions for an analytically tractable manifold model where the statistical model for the data is a mixture of lower dimensional Gaussians embedded in higher dimensional space. We compute the so-called speciation and collapse transition times, as a function of the ratio of manifold-to-ambient space dimensions, and other characteristics of the data model. An important tool used in our analysis is the exact formula for the mutual information (or free energy) of Generalized Linear Models. |
| title | Analysis of Diffusion Models for Manifold Data |
| topic | Statistics Theory Disordered Systems and Neural Networks Information Theory Machine Learning Probability |
| url | https://arxiv.org/abs/2502.04339 |