Nearest Neighbour Score Estimators for Diffusion Generative Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929423298592768 |
|---|---|
| author | Niedoba, Matthew Green, Dylan Naderiparizi, Saeid Lioutas, Vasileios Lavington, Jonathan Wilder Liang, Xiaoxuan Liu, Yunpeng Zhang, Ke Dabiri, Setareh Ścibior, Adam Zwartsenberg, Berend Wood, Frank |
| author_facet | Niedoba, Matthew Green, Dylan Naderiparizi, Saeid Lioutas, Vasileios Lavington, Jonathan Wilder Liang, Xiaoxuan Liu, Yunpeng Zhang, Ke Dabiri, Setareh Ścibior, Adam Zwartsenberg, Berend Wood, Frank |
| contents | Score function estimation is the cornerstone of both training and sampling from diffusion generative models. Despite this fact, the most commonly used estimators are either biased neural network approximations or high variance Monte Carlo estimators based on the conditional score. We introduce a novel nearest neighbour score function estimator which utilizes multiple samples from the training set to dramatically decrease estimator variance. We leverage our low variance estimator in two compelling applications. Training consistency models with our estimator, we report a significant increase in both convergence speed and sample quality. In diffusion models, we show that our estimator can replace a learned network for probability-flow ODE integration, opening promising new avenues of future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08018 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nearest Neighbour Score Estimators for Diffusion Generative Models Niedoba, Matthew Green, Dylan Naderiparizi, Saeid Lioutas, Vasileios Lavington, Jonathan Wilder Liang, Xiaoxuan Liu, Yunpeng Zhang, Ke Dabiri, Setareh Ścibior, Adam Zwartsenberg, Berend Wood, Frank Machine Learning Computer Vision and Pattern Recognition Score function estimation is the cornerstone of both training and sampling from diffusion generative models. Despite this fact, the most commonly used estimators are either biased neural network approximations or high variance Monte Carlo estimators based on the conditional score. We introduce a novel nearest neighbour score function estimator which utilizes multiple samples from the training set to dramatically decrease estimator variance. We leverage our low variance estimator in two compelling applications. Training consistency models with our estimator, we report a significant increase in both convergence speed and sample quality. In diffusion models, we show that our estimator can replace a learned network for probability-flow ODE integration, opening promising new avenues of future research. |
| title | Nearest Neighbour Score Estimators for Diffusion Generative Models |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.08018 |