Improved Sample Complexity Bounds for Diffusion Model Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911270851051520 |
|---|---|
| author | Gupta, Shivam Parulekar, Aditya Price, Eric Xun, Zhiyang |
| author_facet | Gupta, Shivam Parulekar, Aditya Price, Eric Xun, Zhiyang |
| contents | Diffusion models have become the most popular approach to deep generative modeling of images, largely due to their empirical performance and reliability. From a theoretical standpoint, a number of recent works have studied the iteration complexity of sampling, assuming access to an accurate diffusion model. In this work, we focus on understanding the sample complexity of training such a model; how many samples are needed to learn an accurate diffusion model using a sufficiently expressive neural network? Prior work showed bounds polynomial in the dimension, desired Total Variation error, and Wasserstein error. We show an exponential improvement in the dependence on Wasserstein error and depth, along with improved dependencies on other relevant parameters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_13745 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Improved Sample Complexity Bounds for Diffusion Model Training Gupta, Shivam Parulekar, Aditya Price, Eric Xun, Zhiyang Machine Learning Computer Vision and Pattern Recognition Information Theory Statistics Theory Diffusion models have become the most popular approach to deep generative modeling of images, largely due to their empirical performance and reliability. From a theoretical standpoint, a number of recent works have studied the iteration complexity of sampling, assuming access to an accurate diffusion model. In this work, we focus on understanding the sample complexity of training such a model; how many samples are needed to learn an accurate diffusion model using a sufficiently expressive neural network? Prior work showed bounds polynomial in the dimension, desired Total Variation error, and Wasserstein error. We show an exponential improvement in the dependence on Wasserstein error and depth, along with improved dependencies on other relevant parameters. |
| title | Improved Sample Complexity Bounds for Diffusion Model Training |
| topic | Machine Learning Computer Vision and Pattern Recognition Information Theory Statistics Theory |
| url | https://arxiv.org/abs/2311.13745 |