Stochastic smoothing accelerated gradient method for general constrained nonsmooth convex composite optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912864401358848 |
|---|---|
| author | Wang, Ruyu Zhang, Chao |
| author_facet | Wang, Ruyu Zhang, Chao |
| contents | We propose a novel stochastic smoothing accelerated gradient (SSAG) method for general constrained nonsmooth convex composite optimization, and analyze the convergence rates. The SSAG method allows various smoothing techniques, and can deal with the nonsmooth term that is not easy to compute its proximal term, or that does not own the linear max structure. To the best of our knowledge, it is the first time to develop a stochastic approximation type method that treats the maximization of finite but numerous nonsmooth convex functions as a stochastic function, which significantly improves the computational efficiency. We prove that the SSAG method can simultaneously achieve the best-known order ${\cal{O}}(\frac{1}ε)$ of iteration complexity, and the optimal order ${\cal{O}}(\frac{1}{ε^2})$ of $\cal{SFO}$ complexity, using variable sample-size. Numerical results on the application arising from the distributionally robust optimization demonstrate the effectiveness and efficiency of the proposed SSAG method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_01252 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Stochastic smoothing accelerated gradient method for general constrained nonsmooth convex composite optimization Wang, Ruyu Zhang, Chao Optimization and Control We propose a novel stochastic smoothing accelerated gradient (SSAG) method for general constrained nonsmooth convex composite optimization, and analyze the convergence rates. The SSAG method allows various smoothing techniques, and can deal with the nonsmooth term that is not easy to compute its proximal term, or that does not own the linear max structure. To the best of our knowledge, it is the first time to develop a stochastic approximation type method that treats the maximization of finite but numerous nonsmooth convex functions as a stochastic function, which significantly improves the computational efficiency. We prove that the SSAG method can simultaneously achieve the best-known order ${\cal{O}}(\frac{1}ε)$ of iteration complexity, and the optimal order ${\cal{O}}(\frac{1}{ε^2})$ of $\cal{SFO}$ complexity, using variable sample-size. Numerical results on the application arising from the distributionally robust optimization demonstrate the effectiveness and efficiency of the proposed SSAG method. |
| title | Stochastic smoothing accelerated gradient method for general constrained nonsmooth convex composite optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2308.01252 |