A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911788628443136 |
|---|---|
| author | Zeng, Yuxuan Bai, Jianchao Wang, Shengjia Wang, Zhiguo |
| author_facet | Zeng, Yuxuan Bai, Jianchao Wang, Shengjia Wang, Zhiguo |
| contents | In this paper, we propose a unified framework of inexact stochastic Alternating Direction Method of Multipliers (ADMM) for solving nonconvex problems subject to linear constraints, whose objective comprises an average of finite-sum smooth functions and a nonsmooth but possibly nonconvex function. The new framework is highly versatile. Firstly, it not only covers several existing algorithms such as SADMM, SVRG-ADMM, and SPIDER-ADMM but also guides us to design a novel accelerated hybrid stochastic ADMM algorithm, which utilizes a new hybrid estimator to trade-off variance and bias. Second, it enables us to exploit a more flexible dual stepsize in the convergence analysis. Under some mild conditions, our unified framework preserves $\mathcal{O}(1/T)$ sublinear convergence. Additionally, we establish the linear convergence under error bound conditions. Finally, numerical experiments demonstrate the efficacy of the new algorithm for some nonsmooth and nonconvex problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_02015 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization Zeng, Yuxuan Bai, Jianchao Wang, Shengjia Wang, Zhiguo Optimization and Control In this paper, we propose a unified framework of inexact stochastic Alternating Direction Method of Multipliers (ADMM) for solving nonconvex problems subject to linear constraints, whose objective comprises an average of finite-sum smooth functions and a nonsmooth but possibly nonconvex function. The new framework is highly versatile. Firstly, it not only covers several existing algorithms such as SADMM, SVRG-ADMM, and SPIDER-ADMM but also guides us to design a novel accelerated hybrid stochastic ADMM algorithm, which utilizes a new hybrid estimator to trade-off variance and bias. Second, it enables us to exploit a more flexible dual stepsize in the convergence analysis. Under some mild conditions, our unified framework preserves $\mathcal{O}(1/T)$ sublinear convergence. Additionally, we establish the linear convergence under error bound conditions. Finally, numerical experiments demonstrate the efficacy of the new algorithm for some nonsmooth and nonconvex problems. |
| title | A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2403.02015 |