Probabilistic Iterative Hard Thresholding for Sparse Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908474190856192 |
|---|---|
| author | Bergamaschi, Matteo Cristofari, Andrea Kungurtsev, Vyacheslav Rinaldi, Francesco |
| author_facet | Bergamaschi, Matteo Cristofari, Andrea Kungurtsev, Vyacheslav Rinaldi, Francesco |
| contents | For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the ground truth can be critical in formulating an accurate statistical model. The so-called "l0 norm" which counts the number of non-zero components in a vector, is a strong reliable mechanism of enforcing sparsity when incorporated into an optimization problem for minimizing the fit of a given model to a set of observations. However, in big data settings wherein noisy estimates of the gradient must be evaluated out of computational necessity, the literature is scant on methods that reliably converge. In this paper we present an approach towards solving expectation objective optimization problems with cardinality constraints. We prove convergence of the underlying stochastic process, and demonstrate the performance on two Machine Learning problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_01413 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Probabilistic Iterative Hard Thresholding for Sparse Learning Bergamaschi, Matteo Cristofari, Andrea Kungurtsev, Vyacheslav Rinaldi, Francesco Optimization and Control Machine Learning For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the ground truth can be critical in formulating an accurate statistical model. The so-called "l0 norm" which counts the number of non-zero components in a vector, is a strong reliable mechanism of enforcing sparsity when incorporated into an optimization problem for minimizing the fit of a given model to a set of observations. However, in big data settings wherein noisy estimates of the gradient must be evaluated out of computational necessity, the literature is scant on methods that reliably converge. In this paper we present an approach towards solving expectation objective optimization problems with cardinality constraints. We prove convergence of the underlying stochastic process, and demonstrate the performance on two Machine Learning problems. |
| title | Probabilistic Iterative Hard Thresholding for Sparse Learning |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2409.01413 |