Federated ADMM from Bayesian Duality
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914366729748480 |
|---|---|
| author | Möllenhoff, Thomas Swaroop, Siddharth Doshi-Velez, Finale Khan, Mohammad Emtiyaz |
| author_facet | Möllenhoff, Thomas Swaroop, Siddharth Doshi-Velez, Finale Khan, Mohammad Emtiyaz |
| contents | We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objectives are associated with a duality structure that not only resembles the structure of ADMM's fixed-points but also generalizes it. For example, ADMM-like updates are recovered when the VB objective is optimized over the isotropic-Gaussian family, and new non-trivial extensions are obtained for other exponential-family distributions. These extensions include a Newton-like variant that converges in one step on quadratic objectives and an Adam-like variant that yields up to 7% accuracy boosts for deep heterogeneous cases. Our work opens a new Bayesian way to generalize ADMM and other primal-dual methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13150 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Federated ADMM from Bayesian Duality Möllenhoff, Thomas Swaroop, Siddharth Doshi-Velez, Finale Khan, Mohammad Emtiyaz Machine Learning Optimization and Control We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objectives are associated with a duality structure that not only resembles the structure of ADMM's fixed-points but also generalizes it. For example, ADMM-like updates are recovered when the VB objective is optimized over the isotropic-Gaussian family, and new non-trivial extensions are obtained for other exponential-family distributions. These extensions include a Newton-like variant that converges in one step on quadratic objectives and an Adam-like variant that yields up to 7% accuracy boosts for deep heterogeneous cases. Our work opens a new Bayesian way to generalize ADMM and other primal-dual methods. |
| title | Federated ADMM from Bayesian Duality |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2506.13150 |