An over-relaxed ADMM for separable convex programming and its applications to statistical learning
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914635565760512 |
|---|---|
| author | Ni, Renyuan |
| author_facet | Ni, Renyuan |
| contents | The alternating direction method of multipliers (ADMM) has been applied successfully in a broad spectrum of areas. Moreover, it was shown in the literature that ADMM is closely related to the Douglas-Rachford operator-splitting method, and can be viewed as a special case of the proximal point algorithm (PPA). As is well known, the relaxed version of PPA not only maintains the convergence properties of PPA in theory, but also numerically outperforms the original PPA. Therefore, it is interesting to ask whether ADMM can be accelerated by using the over-relaxation technique. In this paper, we answer this question affirmatively by developing an over-relaxed ADMM integrated with a criterion to decide whether a relaxation step is implemented in the iteration. The global convergence and $O(1/t)$ convergence rate of the over-relaxed ADMM are established. We also implement our proposed algorithm to solve Lasso and sparse inverse covariance selection problems, and compare its performance with the relaxed customized ADMM in \cite{CGHY} and the classical ADMM.
The results show that our algorithm substantially outperforms the other two methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_03651 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An over-relaxed ADMM for separable convex programming and its applications to statistical learning Ni, Renyuan Optimization and Control The alternating direction method of multipliers (ADMM) has been applied successfully in a broad spectrum of areas. Moreover, it was shown in the literature that ADMM is closely related to the Douglas-Rachford operator-splitting method, and can be viewed as a special case of the proximal point algorithm (PPA). As is well known, the relaxed version of PPA not only maintains the convergence properties of PPA in theory, but also numerically outperforms the original PPA. Therefore, it is interesting to ask whether ADMM can be accelerated by using the over-relaxation technique. In this paper, we answer this question affirmatively by developing an over-relaxed ADMM integrated with a criterion to decide whether a relaxation step is implemented in the iteration. The global convergence and $O(1/t)$ convergence rate of the over-relaxed ADMM are established. We also implement our proposed algorithm to solve Lasso and sparse inverse covariance selection problems, and compare its performance with the relaxed customized ADMM in \cite{CGHY} and the classical ADMM. The results show that our algorithm substantially outperforms the other two methods. |
| title | An over-relaxed ADMM for separable convex programming and its applications to statistical learning |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2401.03651 |