Rethinking Hard Thresholding Pursuit: Full Adaptation and Sharp Estimation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913636684922880 |
|---|---|
| author | Zhang, Yanhang Li, Zhifan Liu, Shixiang Wang, Xueqin Yin, Jianxin |
| author_facet | Zhang, Yanhang Li, Zhifan Liu, Shixiang Wang, Xueqin Yin, Jianxin |
| contents | Hard Thresholding Pursuit (HTP) has aroused increasing attention for its robust theoretical guarantees and impressive numerical performance in non-convex optimization. In this paper, we introduce a novel tuning-free procedure, named Full-Adaptive HTP (FAHTP), that simultaneously adapts to both the unknown sparsity and signal strength of the underlying model. We provide an in-depth analysis of the iterative thresholding dynamics of FAHTP, offering refined theoretical insights. In specific, under the beta-min condition $\min_{i \in S^*}|{\boldsymbolβ}^*_i| \ge Cσ(\log p/n)^{1/2}$, we show that the FAHTP achieves oracle estimation rate $σ(s^*/n)^{1/2}$, highlighting its theoretical superiority over convex competitors such as LASSO and SLOPE, and recovers the true support set exactly. More importantly, even without the beta-min condition, our method achieves a tighter error bound than the classical minimax rate with high probability. The comprehensive numerical experiments substantiate our theoretical findings, underscoring the effectiveness and robustness of the proposed FAHTP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02554 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking Hard Thresholding Pursuit: Full Adaptation and Sharp Estimation Zhang, Yanhang Li, Zhifan Liu, Shixiang Wang, Xueqin Yin, Jianxin Statistics Theory Hard Thresholding Pursuit (HTP) has aroused increasing attention for its robust theoretical guarantees and impressive numerical performance in non-convex optimization. In this paper, we introduce a novel tuning-free procedure, named Full-Adaptive HTP (FAHTP), that simultaneously adapts to both the unknown sparsity and signal strength of the underlying model. We provide an in-depth analysis of the iterative thresholding dynamics of FAHTP, offering refined theoretical insights. In specific, under the beta-min condition $\min_{i \in S^*}|{\boldsymbolβ}^*_i| \ge Cσ(\log p/n)^{1/2}$, we show that the FAHTP achieves oracle estimation rate $σ(s^*/n)^{1/2}$, highlighting its theoretical superiority over convex competitors such as LASSO and SLOPE, and recovers the true support set exactly. More importantly, even without the beta-min condition, our method achieves a tighter error bound than the classical minimax rate with high probability. The comprehensive numerical experiments substantiate our theoretical findings, underscoring the effectiveness and robustness of the proposed FAHTP. |
| title | Rethinking Hard Thresholding Pursuit: Full Adaptation and Sharp Estimation |
| topic | Statistics Theory |
| url | https://arxiv.org/abs/2501.02554 |