Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910310139428864 |
|---|---|
| author | Tzikas, Alexandros E. Romao, Licio Pilanci, Mert Abate, Alessandro Kochenderfer, Mykel J. |
| author_facet | Tzikas, Alexandros E. Romao, Licio Pilanci, Mert Abate, Alessandro Kochenderfer, Mykel J. |
| contents | Many machine learning applications require operating on a spatially distributed dataset. Despite technological advances, privacy considerations and communication constraints may prevent gathering the entire dataset in a central unit. In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers, which is commonly used in the optimization literature due to its fast convergence. In contrast to distributed optimization, distributed sampling allows for uncertainty quantification in Bayesian inference tasks. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. For our theoretical results, we use convex optimization tools to establish a fundamental inequality on the generated local sample iterates. This inequality enables us to show convergence of the distribution associated with these iterates to the underlying target distribution in Wasserstein distance. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_15838 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers Tzikas, Alexandros E. Romao, Licio Pilanci, Mert Abate, Alessandro Kochenderfer, Mykel J. Machine Learning Multiagent Systems Optimization and Control Computation Many machine learning applications require operating on a spatially distributed dataset. Despite technological advances, privacy considerations and communication constraints may prevent gathering the entire dataset in a central unit. In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers, which is commonly used in the optimization literature due to its fast convergence. In contrast to distributed optimization, distributed sampling allows for uncertainty quantification in Bayesian inference tasks. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. For our theoretical results, we use convex optimization tools to establish a fundamental inequality on the generated local sample iterates. This inequality enables us to show convergence of the distribution associated with these iterates to the underlying target distribution in Wasserstein distance. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods. |
| title | Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers |
| topic | Machine Learning Multiagent Systems Optimization and Control Computation |
| url | https://arxiv.org/abs/2401.15838 |