Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.13438 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908594521243648 |
|---|---|
| author | Glaser, Pierre Huang, Kevin Han Gretton, Arthur |
| author_facet | Glaser, Pierre Huang, Kevin Han Gretton, Arthur |
| contents | We perform a non-asymptotic analysis of the contrastive divergence (CD) algorithm, a training method for unnormalized models. While prior work has established that (for exponential family distributions) the CD iterates asymptotically converge at an $O(n^{-1 / 3})$ rate to the true parameter of the data distribution, we show, under some regularity assumptions, that CD can achieve the parametric rate $O(n^{-1 / 2})$. Our analysis provides results for various data batching schemes, including the fully online and minibatch ones. We additionally show that CD can be near-optimal, in the sense that its asymptotic variance is close to the Cramér-Rao lower bound. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13438 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Near-Optimality of Contrastive Divergence Algorithms Glaser, Pierre Huang, Kevin Han Gretton, Arthur Machine Learning We perform a non-asymptotic analysis of the contrastive divergence (CD) algorithm, a training method for unnormalized models. While prior work has established that (for exponential family distributions) the CD iterates asymptotically converge at an $O(n^{-1 / 3})$ rate to the true parameter of the data distribution, we show, under some regularity assumptions, that CD can achieve the parametric rate $O(n^{-1 / 2})$. Our analysis provides results for various data batching schemes, including the fully online and minibatch ones. We additionally show that CD can be near-optimal, in the sense that its asymptotic variance is close to the Cramér-Rao lower bound. |
| title | Near-Optimality of Contrastive Divergence Algorithms |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.13438 |