CoNeT-GIANT: A compressed Newton-type fully distributed optimization algorithm
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915573342928896 |
|---|---|
| author | Das, Souvik Dey, Subhrakanti |
| author_facet | Das, Souvik Dey, Subhrakanti |
| contents | Compression techniques are essential in distributed optimization and learning algorithms with high-dimensional model parameters, particularly in scenarios with tight communication constraints such as limited bandwidth. This article presents a communication-efficient second-order distributed optimization algorithm, termed as CoNet-GIANT, equipped with a compression module, designed to minimize the average of local strongly convex functions. CoNet-GIANT incorporates two consensus-based averaging steps at each node: gradient tracking and approximate Newton-type iterations, inspired by the recently proposed Network-GIANT. Under certain sufficient conditions on the step size, CoNet-GIANT achieves significantly faster linear convergence, comparable to that of its first-order counterparts, both in the compressed and uncompressed settings. CoNet-GIANT is efficient in terms of data usage, communication cost, and run-time, making it a suitable choice for distributed optimization over a wide range of wireless networks. Extensive experiments on synthetic data and the widely used CovType dataset demonstrate its superior performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_08806 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CoNeT-GIANT: A compressed Newton-type fully distributed optimization algorithm Das, Souvik Dey, Subhrakanti Optimization and Control Compression techniques are essential in distributed optimization and learning algorithms with high-dimensional model parameters, particularly in scenarios with tight communication constraints such as limited bandwidth. This article presents a communication-efficient second-order distributed optimization algorithm, termed as CoNet-GIANT, equipped with a compression module, designed to minimize the average of local strongly convex functions. CoNet-GIANT incorporates two consensus-based averaging steps at each node: gradient tracking and approximate Newton-type iterations, inspired by the recently proposed Network-GIANT. Under certain sufficient conditions on the step size, CoNet-GIANT achieves significantly faster linear convergence, comparable to that of its first-order counterparts, both in the compressed and uncompressed settings. CoNet-GIANT is efficient in terms of data usage, communication cost, and run-time, making it a suitable choice for distributed optimization over a wide range of wireless networks. Extensive experiments on synthetic data and the widely used CovType dataset demonstrate its superior performance. |
| title | CoNeT-GIANT: A compressed Newton-type fully distributed optimization algorithm |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2510.08806 |