Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.15925 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911453588488192 |
|---|---|
| author | Mensch, Zier Holdijk, Lars Duffield, Samuel Aifer, Maxwell Coles, Patrick J. Welling, Max Cheng, Miranda C. N. |
| author_facet | Mensch, Zier Holdijk, Lars Duffield, Samuel Aifer, Maxwell Coles, Patrick J. Welling, Max Cheng, Miranda C. N. |
| contents | Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretization. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15925 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation Mensch, Zier Holdijk, Lars Duffield, Samuel Aifer, Maxwell Coles, Patrick J. Welling, Max Cheng, Miranda C. N. Machine Learning Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretization. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance. |
| title | Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.15925 |