Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.22855 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918074793328640 |
|---|---|
| author | Doostmohammadian, Mohammadreza Rabiee, Hamid R. |
| author_facet | Doostmohammadian, Mohammadreza Rabiee, Hamid R. |
| contents | Distributed optimization advances centralized machine learning methods by enabling parallel and decentralized learning processes over a network of computing nodes. This work provides an accelerated consensus-based distributed algorithm for locally non-convex optimization using the gradient-tracking technique. The proposed algorithm (i) improves the convergence rate by adding momentum towards the optimal state using the heavy-ball method, while (ii) addressing general sector-bound nonlinearities over the information-sharing network. The link nonlinearity includes any sign-preserving odd sector-bound mapping, for example, log-scale data quantization or clipping in practical applications. For admissible momentum and gradient-tracking parameters, using perturbation theory and eigen-spectrum analysis, we prove convergence even in the presence of sector-bound nonlinearity and for locally non-convex cost functions. Further, in contrast to most existing weight-stochastic algorithms, we adopt weight-balanced (WB) network design. This WB design and perturbation-based analysis allow to handle dynamic directed network of agents to address possible time-varying setups due to link failures or packet drops. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_22855 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Momentum-based Accelerated Algorithm for Distributed Optimization under Sector-Bound Nonlinearity Doostmohammadian, Mohammadreza Rabiee, Hamid R. Systems and Control Distributed, Parallel, and Cluster Computing Multiagent Systems Signal Processing Optimization and Control Distributed optimization advances centralized machine learning methods by enabling parallel and decentralized learning processes over a network of computing nodes. This work provides an accelerated consensus-based distributed algorithm for locally non-convex optimization using the gradient-tracking technique. The proposed algorithm (i) improves the convergence rate by adding momentum towards the optimal state using the heavy-ball method, while (ii) addressing general sector-bound nonlinearities over the information-sharing network. The link nonlinearity includes any sign-preserving odd sector-bound mapping, for example, log-scale data quantization or clipping in practical applications. For admissible momentum and gradient-tracking parameters, using perturbation theory and eigen-spectrum analysis, we prove convergence even in the presence of sector-bound nonlinearity and for locally non-convex cost functions. Further, in contrast to most existing weight-stochastic algorithms, we adopt weight-balanced (WB) network design. This WB design and perturbation-based analysis allow to handle dynamic directed network of agents to address possible time-varying setups due to link failures or packet drops. |
| title | Momentum-based Accelerated Algorithm for Distributed Optimization under Sector-Bound Nonlinearity |
| topic | Systems and Control Distributed, Parallel, and Cluster Computing Multiagent Systems Signal Processing Optimization and Control |
| url | https://arxiv.org/abs/2506.22855 |