Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.18130 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915563939299328 |
|---|---|
| author | Ilin, Vasily Sushko, Peter Hu, Jingwei |
| author_facet | Ilin, Vasily Sushko, Peter Hu, Jingwei |
| contents | We propose a deterministic sampling framework using Score-Based Transport Modeling for sampling an unnormalized target density $π$ given only its score $\nabla \log π$. Our method approximates the Wasserstein gradient flow on $\mathrm{KL}(f_t\|π)$ by learning the time-varying score $\nabla \log f_t$ on the fly using score matching. While having the same marginal distribution as Langevin dynamics, our method produces smooth deterministic trajectories, resulting in monotone noise-free convergence. We prove that our method dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. Numerical experiments validate our theoretical findings: our method converges at the optimal rate, has smooth trajectories, and is often more sample efficient than its stochastic counterpart. Experiments on high-dimensional image data show that our method produces high-quality generations in as few as 15 steps and exhibits natural exploratory behavior. The memory and runtime scale linearly in the sample size. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18130 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Score-based deterministic density sampling Ilin, Vasily Sushko, Peter Hu, Jingwei Machine Learning Probability Statistics Theory Primary 65C05, 35Q84, 49Q22, Secondary 60H10, 68T07 We propose a deterministic sampling framework using Score-Based Transport Modeling for sampling an unnormalized target density $π$ given only its score $\nabla \log π$. Our method approximates the Wasserstein gradient flow on $\mathrm{KL}(f_t\|π)$ by learning the time-varying score $\nabla \log f_t$ on the fly using score matching. While having the same marginal distribution as Langevin dynamics, our method produces smooth deterministic trajectories, resulting in monotone noise-free convergence. We prove that our method dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. Numerical experiments validate our theoretical findings: our method converges at the optimal rate, has smooth trajectories, and is often more sample efficient than its stochastic counterpart. Experiments on high-dimensional image data show that our method produces high-quality generations in as few as 15 steps and exhibits natural exploratory behavior. The memory and runtime scale linearly in the sample size. |
| title | Score-based deterministic density sampling |
| topic | Machine Learning Probability Statistics Theory Primary 65C05, 35Q84, 49Q22, Secondary 60H10, 68T07 |
| url | https://arxiv.org/abs/2504.18130 |