Efficient Gradient Tracking Algorithms for Distributed Optimization Problems with Inexact Communication
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914362210385920 |
|---|---|
| author | Zhao, Shengchao Liu, Yongchao |
| author_facet | Zhao, Shengchao Liu, Yongchao |
| contents | Distributed optimization problems usually face inexact communication issues induced by channel noise, communication quantization or differential privacy protection. Most existing algorithms need a two-timescale setting of the stepsize of gradient descent and the parameter of noise suppression to ensure the convergence to the optimal solution. In this paper, we propose two single-timescale algorithms, VRA-DGT and VRA-DSGT, for distributed deterministic and stochastic optimization problems with inexact communication respectively. VRA-DGT integrates the Variance-Reduced Aggregation (VRA) mechanism with the distributed gradient tracking framework, which achieves the convergence rate of $\mathcal{O}\left(k^{-1}\right)$ in the mean square sense and $\mathcal{O}\left(\frac{\ln(k+1)}{k^b}\right)$, $\forall b\in(0.5,1)$ in the almost sure sense when the objective function is strongly convex and smooth. For stochastic optimization problems, VRA-DSGT, where a hybrid variance-reduced technique has been introduced in VRA-DGT, maintains the convergence rate of $\mathcal{O}\left(k^{-1}\right)$ in the mean square sense and $\mathcal{O}\left(\frac{\ln(k+1)}{k^b}\right)$, $\forall b\in(0.5,1)$ in the almost sure sense. Simulated experiments on a logistic regression problem with real-world data verify the effectiveness of the proposed algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05737 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Gradient Tracking Algorithms for Distributed Optimization Problems with Inexact Communication Zhao, Shengchao Liu, Yongchao Optimization and Control Distributed optimization problems usually face inexact communication issues induced by channel noise, communication quantization or differential privacy protection. Most existing algorithms need a two-timescale setting of the stepsize of gradient descent and the parameter of noise suppression to ensure the convergence to the optimal solution. In this paper, we propose two single-timescale algorithms, VRA-DGT and VRA-DSGT, for distributed deterministic and stochastic optimization problems with inexact communication respectively. VRA-DGT integrates the Variance-Reduced Aggregation (VRA) mechanism with the distributed gradient tracking framework, which achieves the convergence rate of $\mathcal{O}\left(k^{-1}\right)$ in the mean square sense and $\mathcal{O}\left(\frac{\ln(k+1)}{k^b}\right)$, $\forall b\in(0.5,1)$ in the almost sure sense when the objective function is strongly convex and smooth. For stochastic optimization problems, VRA-DSGT, where a hybrid variance-reduced technique has been introduced in VRA-DGT, maintains the convergence rate of $\mathcal{O}\left(k^{-1}\right)$ in the mean square sense and $\mathcal{O}\left(\frac{\ln(k+1)}{k^b}\right)$, $\forall b\in(0.5,1)$ in the almost sure sense. Simulated experiments on a logistic regression problem with real-world data verify the effectiveness of the proposed algorithms. |
| title | Efficient Gradient Tracking Algorithms for Distributed Optimization Problems with Inexact Communication |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2501.05737 |