Momentum-based gradient descent methods for Lie groups
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915419079573504 |
|---|---|
| author | Campos, Cédric M. de Diego, David Martín Torrente, José |
| author_facet | Campos, Cédric M. de Diego, David Martín Torrente, José |
| contents | Polyak's Heavy Ball (PHB; Polyak, 1964), a.k.a. Classical Momentum, and Nesterov's Accelerated Gradient (NAG; Nesterov, 1983) are well-established momentum-descent methods for optimization. Although the latter generally outperforms the former, primarily, generalizations of PHB-like methods to nonlinear spaces have not been sufficiently explored in the literature. In this paper, we propose a generalization of NAG-like methods for Lie group optimization. This generalization is based on the variational one-to-one correspondence between classical and accelerated momentum methods (Campos et al., 2023). We provide numerical experiments for chosen retractions on the group of rotations based on the Frobenius norm and the Rosenbrock function to demonstrate the effectiveness of our proposed methods, and that align with results of the Euclidean case, that is, a faster convergence rate for NAG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_09363 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Momentum-based gradient descent methods for Lie groups Campos, Cédric M. de Diego, David Martín Torrente, José Optimization and Control Machine Learning Numerical Analysis Differential Geometry 65K10 (Primary) 70G45, 22E99 (Secondary) Polyak's Heavy Ball (PHB; Polyak, 1964), a.k.a. Classical Momentum, and Nesterov's Accelerated Gradient (NAG; Nesterov, 1983) are well-established momentum-descent methods for optimization. Although the latter generally outperforms the former, primarily, generalizations of PHB-like methods to nonlinear spaces have not been sufficiently explored in the literature. In this paper, we propose a generalization of NAG-like methods for Lie group optimization. This generalization is based on the variational one-to-one correspondence between classical and accelerated momentum methods (Campos et al., 2023). We provide numerical experiments for chosen retractions on the group of rotations based on the Frobenius norm and the Rosenbrock function to demonstrate the effectiveness of our proposed methods, and that align with results of the Euclidean case, that is, a faster convergence rate for NAG. |
| title | Momentum-based gradient descent methods for Lie groups |
| topic | Optimization and Control Machine Learning Numerical Analysis Differential Geometry 65K10 (Primary) 70G45, 22E99 (Secondary) |
| url | https://arxiv.org/abs/2404.09363 |