Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2301.06769 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929471754338304 |
|---|---|
| author | Li, Lei Liu, Jian-Guo Wang, Yuliang |
| author_facet | Li, Lei Liu, Jian-Guo Wang, Yuliang |
| contents | We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings. Via the technique of reflection coupling, we prove the Wasserstein contraction of SGLD when the target distribution is log-concave only outside some compact set. The time discretization and the minibatch in SGLD introduce several difficulties when applying the reflection coupling, which are addressed by a series of careful estimates of conditional expectations. As a direct corollary, the SGLD with constant step size has an invariant distribution and we are able to obtain its geometric ergodicity in terms of $W_1$ distance. The generalization to non-gradient drifts is also included. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_06769 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Geometric ergodicity of SGLD via reflection coupling Li, Lei Liu, Jian-Guo Wang, Yuliang Probability Machine Learning 60H35, 65C40, 37A25 We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings. Via the technique of reflection coupling, we prove the Wasserstein contraction of SGLD when the target distribution is log-concave only outside some compact set. The time discretization and the minibatch in SGLD introduce several difficulties when applying the reflection coupling, which are addressed by a series of careful estimates of conditional expectations. As a direct corollary, the SGLD with constant step size has an invariant distribution and we are able to obtain its geometric ergodicity in terms of $W_1$ distance. The generalization to non-gradient drifts is also included. |
| title | Geometric ergodicity of SGLD via reflection coupling |
| topic | Probability Machine Learning 60H35, 65C40, 37A25 |
| url | https://arxiv.org/abs/2301.06769 |