Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.08912 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917029592694784 |
|---|---|
| author | Alaoui, Ahmed El Montanari, Andrea Sellke, Mark |
| author_facet | Alaoui, Ahmed El Montanari, Andrea Sellke, Mark |
| contents | We consider Ising mixed $p$-spin glasses at high-temperature and without external field, and study the problem of sampling from the Gibbs distribution $μ$ in polynomial time. We develop a new sampling algorithm with complexity of the same order as evaluating the gradient of the Hamiltonian and, in particular, at most linear in the input size. We prove that, at sufficiently high-temperature, it produces samples from a distribution $μ^{alg}$ which is close in normalized Wasserstein distance to $μ$. Namely, there exists a coupling of $μ$ and $μ^{alg}$ such that if $({\boldsymbol x},{\boldsymbol x}^{alg})\in\{-1,+1\}^n\times \{-1,+1\}^n$ is a pair drawn from this coupling, then $n^{-1}{\mathbb E}\{\|{\boldsymbol x}-{\boldsymbol x}^{alg}\|_2^2\}=o_n(1)$. For the case of the Sherrington-Kirkpatrick model, our algorithm succeeds in the full replica-symmetric phase.
We complement this result with a negative one for sampling algorithms satisfying a certain `stability' property, which is verified by many standard techniques.
No stable algorithm can approximately sample at temperatures below the onset of shattering, even under the normalized Wasserstein metric. Further, no algorithm can sample at temperatures below the onset of replica symmetry breaking.
Our sampling method implements a discretized version of a diffusion process that has become recently popular in machine learning under the name of `denoising diffusion.' We derive the same process from the general construction of stochastic localization. Implementing the diffusion process requires to efficiently approximate the mean of the tilted measure. To this end, we use an approximate message passing algorithm that, as we prove, achieves sufficiently accurate mean estimation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_08912 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Sampling from Mean-Field Gibbs Measures via Diffusion Processes Alaoui, Ahmed El Montanari, Andrea Sellke, Mark Probability We consider Ising mixed $p$-spin glasses at high-temperature and without external field, and study the problem of sampling from the Gibbs distribution $μ$ in polynomial time. We develop a new sampling algorithm with complexity of the same order as evaluating the gradient of the Hamiltonian and, in particular, at most linear in the input size. We prove that, at sufficiently high-temperature, it produces samples from a distribution $μ^{alg}$ which is close in normalized Wasserstein distance to $μ$. Namely, there exists a coupling of $μ$ and $μ^{alg}$ such that if $({\boldsymbol x},{\boldsymbol x}^{alg})\in\{-1,+1\}^n\times \{-1,+1\}^n$ is a pair drawn from this coupling, then $n^{-1}{\mathbb E}\{\|{\boldsymbol x}-{\boldsymbol x}^{alg}\|_2^2\}=o_n(1)$. For the case of the Sherrington-Kirkpatrick model, our algorithm succeeds in the full replica-symmetric phase. We complement this result with a negative one for sampling algorithms satisfying a certain `stability' property, which is verified by many standard techniques. No stable algorithm can approximately sample at temperatures below the onset of shattering, even under the normalized Wasserstein metric. Further, no algorithm can sample at temperatures below the onset of replica symmetry breaking. Our sampling method implements a discretized version of a diffusion process that has become recently popular in machine learning under the name of `denoising diffusion.' We derive the same process from the general construction of stochastic localization. Implementing the diffusion process requires to efficiently approximate the mean of the tilted measure. To this end, we use an approximate message passing algorithm that, as we prove, achieves sufficiently accurate mean estimation. |
| title | Sampling from Mean-Field Gibbs Measures via Diffusion Processes |
| topic | Probability |
| url | https://arxiv.org/abs/2310.08912 |