Mean square error analysis of stochastic gradient and variance-reduced sampling algorithms
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912691403096064 |
|---|---|
| author | Lu, Jianfeng Ye, Xuda Zhou, Zhennan |
| author_facet | Lu, Jianfeng Ye, Xuda Zhou, Zhennan |
| contents | This paper considers mean square error (MSE) analysis for stochastic gradient sampling algorithms applied to underdamped Langevin dynamics under a global convexity assumption. A novel discrete Poisson equation framework is developed to bound the time-averaged sampling error. For the Stochastic Gradient UBU (SG-UBU) sampler, we derive an explicit MSE bound and establish that the numerical bias exhibits first-order convergence with respect to the step size $h$, with the leading error coefficient proportional to the variance of the stochastic gradient. The analysis is further extended to variance-reduced algorithms for finite-sum potentials, specifically the SVRG-UBU and SAGA-UBU methods. For these algorithms, we identify a phase transition phenomenon whereby the convergence rate of the numerical bias shifts from first to second order as the step size decreases below a critical threshold. Theoretical findings are validated by numerical experiments. In addition, the analysis provides a practical empirical criterion for selecting between the mini-batch SG-UBU and SVRG-UBU samplers to achieve optimal computational efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04413 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mean square error analysis of stochastic gradient and variance-reduced sampling algorithms Lu, Jianfeng Ye, Xuda Zhou, Zhennan Numerical Analysis 65C30, 60H35, 62F15 This paper considers mean square error (MSE) analysis for stochastic gradient sampling algorithms applied to underdamped Langevin dynamics under a global convexity assumption. A novel discrete Poisson equation framework is developed to bound the time-averaged sampling error. For the Stochastic Gradient UBU (SG-UBU) sampler, we derive an explicit MSE bound and establish that the numerical bias exhibits first-order convergence with respect to the step size $h$, with the leading error coefficient proportional to the variance of the stochastic gradient. The analysis is further extended to variance-reduced algorithms for finite-sum potentials, specifically the SVRG-UBU and SAGA-UBU methods. For these algorithms, we identify a phase transition phenomenon whereby the convergence rate of the numerical bias shifts from first to second order as the step size decreases below a critical threshold. Theoretical findings are validated by numerical experiments. In addition, the analysis provides a practical empirical criterion for selecting between the mini-batch SG-UBU and SVRG-UBU samplers to achieve optimal computational efficiency. |
| title | Mean square error analysis of stochastic gradient and variance-reduced sampling algorithms |
| topic | Numerical Analysis 65C30, 60H35, 62F15 |
| url | https://arxiv.org/abs/2511.04413 |