Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913800837398528 |
|---|---|
| author | Deng, Kangkang Zhang, Shuchang Wang, Boyu Jin, Jiachen Zhou, Juan Wang, Hongxia |
| author_facet | Deng, Kangkang Zhang, Shuchang Wang, Boyu Jin, Jiachen Zhou, Juan Wang, Hongxia |
| contents | This paper proposes SMADMM, a single-loop Stochastic Momentum Alternating Direction Method of Multipliers for solving a class of nonconvex and nonsmooth composite optimization problems. SMADMM achieves the optimal oracle complexity of $\mathcal{O}(ε^{-3/2})$ in the online setting. Unlike previous stochastic ADMM algorithms that require large mini-batches or a double-loop structure, SMADMM uses only $\mathcal{O}(1)$ stochastic gradient evaluations per iteration and avoids costly restarts. To further improve practicality, we incorporate dynamic step sizes and penalty parameters, proving that SMADMM maintains its optimal complexity without the need for large initial batches. We also develop PnP-SMADMM by integrating plug-and-play priors, and establish its theoretical convergence under mild assumptions. Extensive experiments on classification, CT image reconstruction, and phase retrieval tasks demonstrate that our approach outperforms existing stochastic ADMM methods both in accuracy and efficiency, validating our theoretical results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_08223 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm Deng, Kangkang Zhang, Shuchang Wang, Boyu Jin, Jiachen Zhou, Juan Wang, Hongxia Optimization and Control Numerical Analysis 65K05, 65K10, 90C05, 90C26, 90C30 This paper proposes SMADMM, a single-loop Stochastic Momentum Alternating Direction Method of Multipliers for solving a class of nonconvex and nonsmooth composite optimization problems. SMADMM achieves the optimal oracle complexity of $\mathcal{O}(ε^{-3/2})$ in the online setting. Unlike previous stochastic ADMM algorithms that require large mini-batches or a double-loop structure, SMADMM uses only $\mathcal{O}(1)$ stochastic gradient evaluations per iteration and avoids costly restarts. To further improve practicality, we incorporate dynamic step sizes and penalty parameters, proving that SMADMM maintains its optimal complexity without the need for large initial batches. We also develop PnP-SMADMM by integrating plug-and-play priors, and establish its theoretical convergence under mild assumptions. Extensive experiments on classification, CT image reconstruction, and phase retrieval tasks demonstrate that our approach outperforms existing stochastic ADMM methods both in accuracy and efficiency, validating our theoretical results. |
| title | Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm |
| topic | Optimization and Control Numerical Analysis 65K05, 65K10, 90C05, 90C26, 90C30 |
| url | https://arxiv.org/abs/2504.08223 |