Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909640988557312 |
|---|---|
| author | Takezawa, Yuki Jiang, Xiaowen Rodomanov, Anton Stich, Sebastian U. |
| author_facet | Takezawa, Yuki Jiang, Xiaowen Rodomanov, Anton Stich, Sebastian U. |
| contents | Reducing communication complexity is critical for efficient decentralized optimization. The proximal decentralized optimization (PDO) framework is particularly appealing, as methods within this framework can exploit functional similarity among nodes to reduce communication rounds. Specifically, when local functions at different nodes are similar, these methods achieve faster convergence with fewer communication steps. However, existing PDO methods often require highly accurate solutions to subproblems associated with the proximal operator, resulting in significant computational overhead. In this work, we propose the Stabilized Proximal Decentralized Optimization (SPDO) method, which achieves state-of-the-art communication and computational complexities within the PDO framework. Additionally, we refine the analysis of existing PDO methods by relaxing subproblem accuracy requirements and leveraging average functional similarity. Experimental results demonstrate that SPDO significantly outperforms existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_05791 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization Takezawa, Yuki Jiang, Xiaowen Rodomanov, Anton Stich, Sebastian U. Machine Learning Optimization and Control Reducing communication complexity is critical for efficient decentralized optimization. The proximal decentralized optimization (PDO) framework is particularly appealing, as methods within this framework can exploit functional similarity among nodes to reduce communication rounds. Specifically, when local functions at different nodes are similar, these methods achieve faster convergence with fewer communication steps. However, existing PDO methods often require highly accurate solutions to subproblems associated with the proximal operator, resulting in significant computational overhead. In this work, we propose the Stabilized Proximal Decentralized Optimization (SPDO) method, which achieves state-of-the-art communication and computational complexities within the PDO framework. Additionally, we refine the analysis of existing PDO methods by relaxing subproblem accuracy requirements and leveraging average functional similarity. Experimental results demonstrate that SPDO significantly outperforms existing methods. |
| title | Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2506.05791 |