Coordinate Descent for SLOPE
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909198531428352 |
|---|---|
| author | Larsson, Johan Klopfenstein, Quentin Massias, Mathurin Wallin, Jonas |
| author_facet | Larsson, Johan Klopfenstein, Quentin Massias, Mathurin Wallin, Jonas |
| contents | The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we show that our method outperforms a long list of competing algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_14780 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Coordinate Descent for SLOPE Larsson, Johan Klopfenstein, Quentin Massias, Mathurin Wallin, Jonas Optimization and Control Computation Machine Learning 62-08 G.1.6; G.4; G.3 The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we show that our method outperforms a long list of competing algorithms. |
| title | Coordinate Descent for SLOPE |
| topic | Optimization and Control Computation Machine Learning 62-08 G.1.6; G.4; G.3 |
| url | https://arxiv.org/abs/2210.14780 |