Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.13646 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914329701384192 |
|---|---|
| author | Chen, Jun Zhu, Tianyi Ye, Haishan Liu, Lina Dai, Guang Liu, Yong Jiang, Yunliang Tsang, Ivor W. |
| author_facet | Chen, Jun Zhu, Tianyi Ye, Haishan Liu, Lina Dai, Guang Liu, Yong Jiang, Yunliang Tsang, Ivor W. |
| contents | Gradient descent with momentum has been widely applied in various signal processing and machine learning tasks, demonstrating a notable empirical advantage over standard gradient descent. However, momentum-based distributed Riemannian algorithms have been only scarcely explored. In this paper, we propose Riemannian Momentum Tracking (RMTracking), a decentralized optimization algorithm with momentum over a compact submanifold. Given the non-convex nature of compact submanifolds, the objective function, composed of a finite sum of smooth (possibly non-convex) local functions, is minimized across agents in an undirected and connected network graph. With a constant step-size, we establish an $\mathcal{O}(\frac{1-β}{K})$ convergence rate of the Riemannian gradient average for any momentum weight $β\in [0,1)$. Especially, RMTracking can achieve a convergence rate of $\mathcal{O}(\frac{1-β}{K})$ to a stationary point when the step-size is sufficiently small. To best of our knowledge, RMTracking is the first decentralized algorithm to achieve exact convergence that is $\frac{1}{1-β}$ times faster than other related algorithms. Finally, we verify these theoretical claims through numerical experiments on eigenvalue problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_13646 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Riemannian Momentum Tracking: Distributed Optimization with Momentum on Compact Submanifolds Chen, Jun Zhu, Tianyi Ye, Haishan Liu, Lina Dai, Guang Liu, Yong Jiang, Yunliang Tsang, Ivor W. Optimization and Control Gradient descent with momentum has been widely applied in various signal processing and machine learning tasks, demonstrating a notable empirical advantage over standard gradient descent. However, momentum-based distributed Riemannian algorithms have been only scarcely explored. In this paper, we propose Riemannian Momentum Tracking (RMTracking), a decentralized optimization algorithm with momentum over a compact submanifold. Given the non-convex nature of compact submanifolds, the objective function, composed of a finite sum of smooth (possibly non-convex) local functions, is minimized across agents in an undirected and connected network graph. With a constant step-size, we establish an $\mathcal{O}(\frac{1-β}{K})$ convergence rate of the Riemannian gradient average for any momentum weight $β\in [0,1)$. Especially, RMTracking can achieve a convergence rate of $\mathcal{O}(\frac{1-β}{K})$ to a stationary point when the step-size is sufficiently small. To best of our knowledge, RMTracking is the first decentralized algorithm to achieve exact convergence that is $\frac{1}{1-β}$ times faster than other related algorithms. Finally, we verify these theoretical claims through numerical experiments on eigenvalue problems. |
| title | Riemannian Momentum Tracking: Distributed Optimization with Momentum on Compact Submanifolds |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2602.13646 |