A prediction rigidity formalism for low-cost uncertainties in trained neural networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929264191864832 |
|---|---|
| author | Bigi, Filippo Chong, Sanggyu Ceriotti, Michele Grasselli, Federico |
| author_facet | Bigi, Filippo Chong, Sanggyu Ceriotti, Michele Grasselli, Federico |
| contents | Regression methods are fundamental for scientific and technological applications. However, fitted models can be highly unreliable outside of their training domain, and hence the quantification of their uncertainty is crucial in many of their applications. Based on the solution of a constrained optimization problem, we propose "prediction rigidities" as a method to obtain uncertainties of arbitrary pre-trained regressors. We establish a strong connection between our framework and Bayesian inference, and we develop a last-layer approximation that allows the new method to be applied to neural networks. This extension affords cheap uncertainties without any modification to the neural network itself or its training procedure. We show the effectiveness of our method on a wide range of regression tasks, ranging from simple toy models to applications in chemistry and meteorology. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_02251 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A prediction rigidity formalism for low-cost uncertainties in trained neural networks Bigi, Filippo Chong, Sanggyu Ceriotti, Michele Grasselli, Federico Machine Learning Regression methods are fundamental for scientific and technological applications. However, fitted models can be highly unreliable outside of their training domain, and hence the quantification of their uncertainty is crucial in many of their applications. Based on the solution of a constrained optimization problem, we propose "prediction rigidities" as a method to obtain uncertainties of arbitrary pre-trained regressors. We establish a strong connection between our framework and Bayesian inference, and we develop a last-layer approximation that allows the new method to be applied to neural networks. This extension affords cheap uncertainties without any modification to the neural network itself or its training procedure. We show the effectiveness of our method on a wide range of regression tasks, ranging from simple toy models to applications in chemistry and meteorology. |
| title | A prediction rigidity formalism for low-cost uncertainties in trained neural networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2403.02251 |