ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909541909659648 |
|---|---|
| author | Schwencke, Nilo Furtlehner, Cyril |
| author_facet | Schwencke, Nilo Furtlehner, Cyril |
| contents | In the recent years, Physics Informed Neural Networks (PINNs) have received strong interest as a method to solve PDE driven systems, in particular for data assimilation purpose. This method is still in its infancy, with many shortcomings and failures that remain not properly understood. In this paper we propose a natural gradient approach to PINNs which contributes to speed-up and improve the accuracy of the training. Based on an in depth analysis of the differential geometric structures of the problem, we come up with two distinct contributions: (i) a new natural gradient algorithm that scales as $\min(P^2S, S^2P)$, where $P$ is the number of parameters, and $S$ the batch size; (ii) a mathematically principled reformulation of the PINNs problem that allows the extension of natural gradient to it, with proved connections to Green's function theory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_10782 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning Schwencke, Nilo Furtlehner, Cyril Machine Learning Artificial Intelligence Numerical Analysis Optimization and Control I.2.8 In the recent years, Physics Informed Neural Networks (PINNs) have received strong interest as a method to solve PDE driven systems, in particular for data assimilation purpose. This method is still in its infancy, with many shortcomings and failures that remain not properly understood. In this paper we propose a natural gradient approach to PINNs which contributes to speed-up and improve the accuracy of the training. Based on an in depth analysis of the differential geometric structures of the problem, we come up with two distinct contributions: (i) a new natural gradient algorithm that scales as $\min(P^2S, S^2P)$, where $P$ is the number of parameters, and $S$ the batch size; (ii) a mathematically principled reformulation of the PINNs problem that allows the extension of natural gradient to it, with proved connections to Green's function theory. |
| title | ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning |
| topic | Machine Learning Artificial Intelligence Numerical Analysis Optimization and Control I.2.8 |
| url | https://arxiv.org/abs/2412.10782 |