Stochastic optimization over expectation-formulated generalized Stiefel manifold
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929651135283200 |
|---|---|
| author | Jiang, Linshuo Xiao, Nachuan Liu, Xin |
| author_facet | Jiang, Linshuo Xiao, Nachuan Liu, Xin |
| contents | In this paper, we consider a class of stochastic optimization problems over the expectation-formulated generalized Stiefel manifold (SOEGS), where the objective function $f$ is continuously differentiable. We propose a novel constraint dissolving penalty function with a customized penalty term (CDFDP), which maintains the same order of differentiability as $f$. Our theoretical analysis establishes the global equivalence between CDFCP and SOEGS in the sense that they share the same first-order and second-order stationary points under mild conditions. These results on equivalence enable the direct implementation of various stochastic optimization approaches to solve SOEGS. In particular, we develop a stochastic gradient algorithm and its accelerated variant by incorporating an adaptive step size strategy. Furthermore, we prove their $\mathcal{O}(\varepsilon^{-4})$ sample complexity for finding an $\varepsilon$-stationary point of CDFCP. Comprehensive numerical experiments show the efficiency and robustness of our proposed algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20008 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Stochastic optimization over expectation-formulated generalized Stiefel manifold Jiang, Linshuo Xiao, Nachuan Liu, Xin Optimization and Control In this paper, we consider a class of stochastic optimization problems over the expectation-formulated generalized Stiefel manifold (SOEGS), where the objective function $f$ is continuously differentiable. We propose a novel constraint dissolving penalty function with a customized penalty term (CDFDP), which maintains the same order of differentiability as $f$. Our theoretical analysis establishes the global equivalence between CDFCP and SOEGS in the sense that they share the same first-order and second-order stationary points under mild conditions. These results on equivalence enable the direct implementation of various stochastic optimization approaches to solve SOEGS. In particular, we develop a stochastic gradient algorithm and its accelerated variant by incorporating an adaptive step size strategy. Furthermore, we prove their $\mathcal{O}(\varepsilon^{-4})$ sample complexity for finding an $\varepsilon$-stationary point of CDFCP. Comprehensive numerical experiments show the efficiency and robustness of our proposed algorithms. |
| title | Stochastic optimization over expectation-formulated generalized Stiefel manifold |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2412.20008 |