Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917329043980288 |
|---|---|
| author | Lerman, Gilad Li, Kang Maunu, Tyler Zhang, Teng |
| author_facet | Lerman, Gilad Li, Kang Maunu, Tyler Zhang, Teng |
| contents | Robust subspace estimation is fundamental to many machine learning and data analysis tasks. Iteratively Reweighted Least Squares (IRLS) is an elegant and empirically effective approach to this problem, yet its theoretical properties remain poorly understood. This paper establishes that, under deterministic conditions, a variant of IRLS with dynamic smoothing regularization converges linearly to the underlying subspace from any initialization. We extend these guarantees to affine subspace estimation, a setting that lacks prior recovery theory. Additionally, we illustrate the practical benefits of IRLS through an application to low-dimensional neural network training. Our results provide the first global convergence guarantees for IRLS in robust subspace recovery and, more broadly, for nonconvex IRLS on a Riemannian manifold. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20533 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery Lerman, Gilad Li, Kang Maunu, Tyler Zhang, Teng Machine Learning Optimization and Control Robust subspace estimation is fundamental to many machine learning and data analysis tasks. Iteratively Reweighted Least Squares (IRLS) is an elegant and empirically effective approach to this problem, yet its theoretical properties remain poorly understood. This paper establishes that, under deterministic conditions, a variant of IRLS with dynamic smoothing regularization converges linearly to the underlying subspace from any initialization. We extend these guarantees to affine subspace estimation, a setting that lacks prior recovery theory. Additionally, we illustrate the practical benefits of IRLS through an application to low-dimensional neural network training. Our results provide the first global convergence guarantees for IRLS in robust subspace recovery and, more broadly, for nonconvex IRLS on a Riemannian manifold. |
| title | Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2506.20533 |