A phase transition in sampling from Restricted Boltzmann Machines
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914969951404032 |
|---|---|
| author | Kwon, Youngwoo Qin, Qian Wang, Guanyang Wei, Yuchen |
| author_facet | Kwon, Youngwoo Qin, Qian Wang, Guanyang Wei, Yuchen |
| contents | Restricted Boltzmann Machines are a class of undirected graphical models that play a key role in deep learning and unsupervised learning. In this study, we prove a phase transition phenomenon in the mixing time of the Gibbs sampler for a one-parameter Restricted Boltzmann Machine. Specifically, the mixing time varies logarithmically, polynomially, and exponentially with the number of vertices depending on whether the parameter $c$ is above, equal to, or below a critical value $c_\star\approx-5.87$. A key insight from our analysis is the link between the Gibbs sampler and a dynamical system, which we utilize to quantify the former based on the behavior of the latter. To study the critical case $c= c_\star$, we develop a new isoperimetric inequality for the sampler's stationary distribution by showing that the distribution is nearly log-concave. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08423 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A phase transition in sampling from Restricted Boltzmann Machines Kwon, Youngwoo Qin, Qian Wang, Guanyang Wei, Yuchen Machine Learning Statistical Mechanics Mathematical Physics Probability Computation Restricted Boltzmann Machines are a class of undirected graphical models that play a key role in deep learning and unsupervised learning. In this study, we prove a phase transition phenomenon in the mixing time of the Gibbs sampler for a one-parameter Restricted Boltzmann Machine. Specifically, the mixing time varies logarithmically, polynomially, and exponentially with the number of vertices depending on whether the parameter $c$ is above, equal to, or below a critical value $c_\star\approx-5.87$. A key insight from our analysis is the link between the Gibbs sampler and a dynamical system, which we utilize to quantify the former based on the behavior of the latter. To study the critical case $c= c_\star$, we develop a new isoperimetric inequality for the sampler's stationary distribution by showing that the distribution is nearly log-concave. |
| title | A phase transition in sampling from Restricted Boltzmann Machines |
| topic | Machine Learning Statistical Mechanics Mathematical Physics Probability Computation |
| url | https://arxiv.org/abs/2410.08423 |