Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Them
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917033970499584 |
|---|---|
| author | Roy, Hrittik Hauberg, Søren Krämer, Nicholas |
| author_facet | Roy, Hrittik Hauberg, Søren Krämer, Nicholas |
| contents | This paper argues that the method of least squares has significant unfulfilled potential in modern machine learning, far beyond merely being a tool for fitting linear models. To release its potential, we derive custom gradients that transform the solver into a differentiable operator, like a neural network layer, enabling many diverse applications. Empirically, we demonstrate: (i) scalability by enforcing weight sparsity on a 50 million parameter model; (ii) imposing conservativeness constraints in score-based generative models; and (iii) hyperparameter tuning of Gaussian processes based on predictive performance. By doing this, our work represents the next iteration in developing differentiable linear-algebra tools and making them widely accessible to machine learning practitioners. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_19634 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Them Roy, Hrittik Hauberg, Søren Krämer, Nicholas Machine Learning Numerical Analysis This paper argues that the method of least squares has significant unfulfilled potential in modern machine learning, far beyond merely being a tool for fitting linear models. To release its potential, we derive custom gradients that transform the solver into a differentiable operator, like a neural network layer, enabling many diverse applications. Empirically, we demonstrate: (i) scalability by enforcing weight sparsity on a 50 million parameter model; (ii) imposing conservativeness constraints in score-based generative models; and (iii) hyperparameter tuning of Gaussian processes based on predictive performance. By doing this, our work represents the next iteration in developing differentiable linear-algebra tools and making them widely accessible to machine learning practitioners. |
| title | Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Them |
| topic | Machine Learning Numerical Analysis |
| url | https://arxiv.org/abs/2510.19634 |