Decentralized Inexact Cubic Newton Method with Consensus Procedure
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917518933753856 |
|---|---|
| author | Agafonov, Artem Novitskii, Anton Rogozin, Alexander Sokolov, Yury Kamzolov, Dmitry Dyakonov, Alexander Takáč, Martin Gasnikov, Alexander |
| author_facet | Agafonov, Artem Novitskii, Anton Rogozin, Alexander Sokolov, Yury Kamzolov, Dmitry Dyakonov, Alexander Takáč, Martin Gasnikov, Alexander |
| contents | Distributed optimization is widely used in large-scale and privacy-preserving machine learning, where each agent stores a local objective and communicates only with its neighbors in a connected network. We study decentralized second-order optimization and focus on consensus procedures that approximately average local iterates, gradients, and Hessians through neighbor-to-neighbor communications. We propose a general Decentralized Cubic Newton method for convex optimization under $L_1$-smoothness of gradients and $L_2$-Lipschitz continuity of Hessians, and develop a theory that accurately tracks the inaccuracies caused by consensus and by disagreement between local iterates. Under these assumptions, the method matches the iteration complexity of the exact Cubic Newton method and requires only additional polylogarithmic communication-round overhead to reach the necessary consensus accuracy. We further propose an Accelerated Decentralized Cubic Newton method for strongly convex objectives and show that it matches the iteration complexity of the exact Accelerated Cubic Newton method, again with only additional polylogarithmic communication-round overhead. Finally, although the general method requires exchanging full $d \times d$ Hessian matrices, we show how it can be implemented for generalized linear models by transmitting only vectors, making the approach substantially more practical in high dimensions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_21169 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Decentralized Inexact Cubic Newton Method with Consensus Procedure Agafonov, Artem Novitskii, Anton Rogozin, Alexander Sokolov, Yury Kamzolov, Dmitry Dyakonov, Alexander Takáč, Martin Gasnikov, Alexander Optimization and Control Distributed optimization is widely used in large-scale and privacy-preserving machine learning, where each agent stores a local objective and communicates only with its neighbors in a connected network. We study decentralized second-order optimization and focus on consensus procedures that approximately average local iterates, gradients, and Hessians through neighbor-to-neighbor communications. We propose a general Decentralized Cubic Newton method for convex optimization under $L_1$-smoothness of gradients and $L_2$-Lipschitz continuity of Hessians, and develop a theory that accurately tracks the inaccuracies caused by consensus and by disagreement between local iterates. Under these assumptions, the method matches the iteration complexity of the exact Cubic Newton method and requires only additional polylogarithmic communication-round overhead to reach the necessary consensus accuracy. We further propose an Accelerated Decentralized Cubic Newton method for strongly convex objectives and show that it matches the iteration complexity of the exact Accelerated Cubic Newton method, again with only additional polylogarithmic communication-round overhead. Finally, although the general method requires exchanging full $d \times d$ Hessian matrices, we show how it can be implemented for generalized linear models by transmitting only vectors, making the approach substantially more practical in high dimensions. |
| title | Decentralized Inexact Cubic Newton Method with Consensus Procedure |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2605.21169 |