Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2207.09304 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913745211490304 |
|---|---|
| author | Li, Lei Wang, Yuliang |
| author_facet | Li, Lei Wang, Yuliang |
| contents | We establish a sharp uniform-in-time error estimate for the Stochastic Gradient Langevin Dynamics (SGLD), which is a widely-used sampling algorithm. Under mild assumptions, we obtain a uniform-in-time $O(η^2)$ bound for the KL-divergence between the SGLD iteration and the Langevin diffusion, where $η$ is the step size (or learning rate). Our analysis is also valid for varying step sizes. Consequently, we are able to derive an $O(η)$ bound for the distance between the invariant measures of the SGLD iteration and the Langevin diffusion, in terms of Wasserstein or total variation distances. Our result can be viewed as a significant improvement compared with existing analysis for SGLD in related literature. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2207_09304 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamics Li, Lei Wang, Yuliang Probability Machine Learning 65C20, 68Q25, 60H30 We establish a sharp uniform-in-time error estimate for the Stochastic Gradient Langevin Dynamics (SGLD), which is a widely-used sampling algorithm. Under mild assumptions, we obtain a uniform-in-time $O(η^2)$ bound for the KL-divergence between the SGLD iteration and the Langevin diffusion, where $η$ is the step size (or learning rate). Our analysis is also valid for varying step sizes. Consequently, we are able to derive an $O(η)$ bound for the distance between the invariant measures of the SGLD iteration and the Langevin diffusion, in terms of Wasserstein or total variation distances. Our result can be viewed as a significant improvement compared with existing analysis for SGLD in related literature. |
| title | A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamics |
| topic | Probability Machine Learning 65C20, 68Q25, 60H30 |
| url | https://arxiv.org/abs/2207.09304 |