Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.01944 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908573456400384 |
|---|---|
| author | Oliva, Paul Felix Valsecchi Akyildiz, O. Deniz Duncan, Andrew |
| author_facet | Oliva, Paul Felix Valsecchi Akyildiz, O. Deniz Duncan, Andrew |
| contents | We propose a continuous-time formulation of persistent contrastive divergence (PCD) for maximum likelihood estimation (MLE) of unnormalised densities. Our approach expresses PCD as a coupled, multiscale system of stochastic differential equations (SDEs), which perform optimisation of the parameter and sampling of the associated parametrised density, simultaneously.
From this novel formulation, we are able to derive explicit bounds for the error between the PCD iterates and the MLE solution for the model parameter. This is made possible by deriving uniform-in-time (UiT) bounds for the difference in moments between the multiscale system and the averaged regime. An efficient implementation of the continuous-time scheme is introduced, leveraging a class of explicit, stable intregators, stochastic orthogonal Runge-Kutta Chebyshev (S-ROCK), for which we provide explicit error estimates in the long-time regime. This leads to a novel method for training energy-based models (EBMs) with explicit error guarantees. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01944 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Uniform-in-time convergence bounds for Persistent Contrastive Divergence Algorithms Oliva, Paul Felix Valsecchi Akyildiz, O. Deniz Duncan, Andrew Machine Learning 68T07, 60J60, 62M05, 60H35 We propose a continuous-time formulation of persistent contrastive divergence (PCD) for maximum likelihood estimation (MLE) of unnormalised densities. Our approach expresses PCD as a coupled, multiscale system of stochastic differential equations (SDEs), which perform optimisation of the parameter and sampling of the associated parametrised density, simultaneously. From this novel formulation, we are able to derive explicit bounds for the error between the PCD iterates and the MLE solution for the model parameter. This is made possible by deriving uniform-in-time (UiT) bounds for the difference in moments between the multiscale system and the averaged regime. An efficient implementation of the continuous-time scheme is introduced, leveraging a class of explicit, stable intregators, stochastic orthogonal Runge-Kutta Chebyshev (S-ROCK), for which we provide explicit error estimates in the long-time regime. This leads to a novel method for training energy-based models (EBMs) with explicit error guarantees. |
| title | Uniform-in-time convergence bounds for Persistent Contrastive Divergence Algorithms |
| topic | Machine Learning 68T07, 60J60, 62M05, 60H35 |
| url | https://arxiv.org/abs/2510.01944 |