Regularized Stein Variational Gradient Flow
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916239589244928 |
|---|---|
| author | He, Ye Balasubramanian, Krishnakumar Sriperumbudur, Bharath K. Lu, Jianfeng |
| author_facet | He, Ye Balasubramanian, Krishnakumar Sriperumbudur, Bharath K. Lu, Jianfeng |
| contents | The Stein Variational Gradient Descent (SVGD) algorithm is a deterministic particle method for sampling. However, a mean-field analysis reveals that the gradient flow corresponding to the SVGD algorithm (i.e., the Stein Variational Gradient Flow) only provides a constant-order approximation to the Wasserstein Gradient Flow corresponding to the KL-divergence minimization. In this work, we propose the Regularized Stein Variational Gradient Flow, which interpolates between the Stein Variational Gradient Flow and the Wasserstein Gradient Flow. We establish various theoretical properties of the Regularized Stein Variational Gradient Flow (and its time-discretization) including convergence to equilibrium, existence and uniqueness of weak solutions, and stability of the solutions. We provide preliminary numerical evidence of the improved performance offered by the regularization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_07861 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Regularized Stein Variational Gradient Flow He, Ye Balasubramanian, Krishnakumar Sriperumbudur, Bharath K. Lu, Jianfeng Machine Learning Numerical Analysis Analysis of PDEs Statistics Theory Computation The Stein Variational Gradient Descent (SVGD) algorithm is a deterministic particle method for sampling. However, a mean-field analysis reveals that the gradient flow corresponding to the SVGD algorithm (i.e., the Stein Variational Gradient Flow) only provides a constant-order approximation to the Wasserstein Gradient Flow corresponding to the KL-divergence minimization. In this work, we propose the Regularized Stein Variational Gradient Flow, which interpolates between the Stein Variational Gradient Flow and the Wasserstein Gradient Flow. We establish various theoretical properties of the Regularized Stein Variational Gradient Flow (and its time-discretization) including convergence to equilibrium, existence and uniqueness of weak solutions, and stability of the solutions. We provide preliminary numerical evidence of the improved performance offered by the regularization. |
| title | Regularized Stein Variational Gradient Flow |
| topic | Machine Learning Numerical Analysis Analysis of PDEs Statistics Theory Computation |
| url | https://arxiv.org/abs/2211.07861 |