Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.05801 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913339758608384 |
|---|---|
| author | De Ryck, Tim Bonnet, Florent Mishra, Siddhartha de Bézenac, Emmanuel |
| author_facet | De Ryck, Tim Bonnet, Florent Mishra, Siddhartha de Bézenac, Emmanuel |
| contents | In this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs). Our key result is that the difficulty in training these models is closely related to the conditioning of a specific differential operator. This operator, in turn, is associated to the Hermitian square of the differential operator of the underlying PDE. If this operator is ill-conditioned, it results in slow or infeasible training. Therefore, preconditioning this operator is crucial. We employ both rigorous mathematical analysis and empirical evaluations to investigate various strategies, explaining how they better condition this critical operator, and consequently improve training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_05801 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | An operator preconditioning perspective on training in physics-informed machine learning De Ryck, Tim Bonnet, Florent Mishra, Siddhartha de Bézenac, Emmanuel Machine Learning In this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs). Our key result is that the difficulty in training these models is closely related to the conditioning of a specific differential operator. This operator, in turn, is associated to the Hermitian square of the differential operator of the underlying PDE. If this operator is ill-conditioned, it results in slow or infeasible training. Therefore, preconditioning this operator is crucial. We employ both rigorous mathematical analysis and empirical evaluations to investigate various strategies, explaining how they better condition this critical operator, and consequently improve training. |
| title | An operator preconditioning perspective on training in physics-informed machine learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2310.05801 |