Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915530960535552 |
|---|---|
| author | Farghly, Tyler Potaptchik, Peter Howard, Samuel Deligiannidis, George Pidstrigach, Jakiw |
| author_facet | Farghly, Tyler Potaptchik, Peter Howard, Samuel Deligiannidis, George Pidstrigach, Jakiw |
| contents | Diffusion models have achieved state-of-the-art performance, demonstrating remarkable generalisation capabilities across diverse domains. However, the mechanisms underpinning these strong capabilities remain only partially understood. A leading conjecture, based on the manifold hypothesis, attributes this success to their ability to adapt to low-dimensional geometric structure within the data. This work provides evidence for this conjecture, focusing on how such phenomena could result from the formulation of the learning problem through score matching. We inspect the role of implicit regularisation by investigating the effect of smoothing minimisers of the empirical score matching objective. Our theoretical and empirical results confirm that smoothing the score function -- or equivalently, smoothing in the log-density domain -- produces smoothing tangential to the data manifold. In addition, we show that the manifold along which the diffusion model generalises can be controlled by choosing an appropriate smoothing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02305 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive Farghly, Tyler Potaptchik, Peter Howard, Samuel Deligiannidis, George Pidstrigach, Jakiw Machine Learning Artificial Intelligence Statistics Theory Diffusion models have achieved state-of-the-art performance, demonstrating remarkable generalisation capabilities across diverse domains. However, the mechanisms underpinning these strong capabilities remain only partially understood. A leading conjecture, based on the manifold hypothesis, attributes this success to their ability to adapt to low-dimensional geometric structure within the data. This work provides evidence for this conjecture, focusing on how such phenomena could result from the formulation of the learning problem through score matching. We inspect the role of implicit regularisation by investigating the effect of smoothing minimisers of the empirical score matching objective. Our theoretical and empirical results confirm that smoothing the score function -- or equivalently, smoothing in the log-density domain -- produces smoothing tangential to the data manifold. In addition, we show that the manifold along which the diffusion model generalises can be controlled by choosing an appropriate smoothing. |
| title | Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive |
| topic | Machine Learning Artificial Intelligence Statistics Theory |
| url | https://arxiv.org/abs/2510.02305 |