Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909235079544832 |
|---|---|
| author | Padmanabha, Govinda Anantha Fuhg, Jan Niklas Safta, Cosmin Jones, Reese E. Bouklas, Nikolaos |
| author_facet | Padmanabha, Govinda Anantha Fuhg, Jan Niklas Safta, Cosmin Jones, Reese E. Bouklas, Nikolaos |
| contents | Most scientific machine learning (SciML) applications of neural networks involve hundreds to thousands of parameters, and hence, uncertainty quantification for such models is plagued by the curse of dimensionality. Using physical applications, we show that $L_0$ sparsification prior to Stein variational gradient descent ($L_0$+SVGD) is a more robust and efficient means of uncertainty quantification, in terms of computational cost and performance than the direct application of SGVD or projected SGVD methods. Specifically, $L_0$+SVGD demonstrates superior resilience to noise, the ability to perform well in extrapolated regions, and a faster convergence rate to an optimal solution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00761 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models Padmanabha, Govinda Anantha Fuhg, Jan Niklas Safta, Cosmin Jones, Reese E. Bouklas, Nikolaos Machine Learning Computational Engineering, Finance, and Science Most scientific machine learning (SciML) applications of neural networks involve hundreds to thousands of parameters, and hence, uncertainty quantification for such models is plagued by the curse of dimensionality. Using physical applications, we show that $L_0$ sparsification prior to Stein variational gradient descent ($L_0$+SVGD) is a more robust and efficient means of uncertainty quantification, in terms of computational cost and performance than the direct application of SGVD or projected SGVD methods. Specifically, $L_0$+SVGD demonstrates superior resilience to noise, the ability to perform well in extrapolated regions, and a faster convergence rate to an optimal solution. |
| title | Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models |
| topic | Machine Learning Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2407.00761 |