Oracle complexities of augmented Lagrangian methods for nonsmooth manifold optimization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914773400027136 |
|---|---|
| author | Deng, Kangkang Hu, Jiang Wu, Jiayuan Wen, Zaiwen |
| author_facet | Deng, Kangkang Hu, Jiang Wu, Jiayuan Wen, Zaiwen |
| contents | In this paper, we present two novel manifold inexact augmented Lagrangian methods, \textbf{ManIAL} for deterministic settings and \textbf{StoManIAL} for stochastic settings, solving nonsmooth manifold optimization problems. By using the Riemannian gradient method as a subroutine, we establish an $\mathcal{O}(ε^{-3})$ oracle complexity result of \textbf{ManIAL}, matching the best-known complexity result. Our algorithm relies on the careful selection of penalty parameters and the precise control of termination criteria for subproblems. Moreover, for cases where the smooth term follows an expectation form, our proposed \textbf{StoManIAL} utilizes a Riemannian recursive momentum method as a subroutine, and achieves an oracle complexity of $\tilde{\mathcal{O}}(ε^{-3.5})$, which surpasses the best-known $\mathcal{O}(ε^{-4})$ result. Numerical experiments conducted on sparse principal component analysis and sparse canonical correlation analysis demonstrate that our proposed methods outperform an existing method with the previously best-known complexity result. To the best of our knowledge, these are the first complexity results of the augmented Lagrangian methods for solving nonsmooth manifold optimization problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_05121 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Oracle complexities of augmented Lagrangian methods for nonsmooth manifold optimization Deng, Kangkang Hu, Jiang Wu, Jiayuan Wen, Zaiwen Optimization and Control 65K05, 65K10, 90C06, 90C26, 90C30, 90C60 In this paper, we present two novel manifold inexact augmented Lagrangian methods, \textbf{ManIAL} for deterministic settings and \textbf{StoManIAL} for stochastic settings, solving nonsmooth manifold optimization problems. By using the Riemannian gradient method as a subroutine, we establish an $\mathcal{O}(ε^{-3})$ oracle complexity result of \textbf{ManIAL}, matching the best-known complexity result. Our algorithm relies on the careful selection of penalty parameters and the precise control of termination criteria for subproblems. Moreover, for cases where the smooth term follows an expectation form, our proposed \textbf{StoManIAL} utilizes a Riemannian recursive momentum method as a subroutine, and achieves an oracle complexity of $\tilde{\mathcal{O}}(ε^{-3.5})$, which surpasses the best-known $\mathcal{O}(ε^{-4})$ result. Numerical experiments conducted on sparse principal component analysis and sparse canonical correlation analysis demonstrate that our proposed methods outperform an existing method with the previously best-known complexity result. To the best of our knowledge, these are the first complexity results of the augmented Lagrangian methods for solving nonsmooth manifold optimization problems. |
| title | Oracle complexities of augmented Lagrangian methods for nonsmooth manifold optimization |
| topic | Optimization and Control 65K05, 65K10, 90C06, 90C26, 90C30, 90C60 |
| url | https://arxiv.org/abs/2404.05121 |