Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909168432054272 |
|---|---|
| author | Kopaničáková, Alena Kothari, Hardik Karniadakis, George Em Krause, Rolf |
| author_facet | Kopaničáková, Alena Kothari, Hardik Karniadakis, George Em Krause, Rolf |
| contents | We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_17648 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies Kopaničáková, Alena Kothari, Hardik Karniadakis, George Em Krause, Rolf Numerical Analysis Machine Learning Optimization and Control 90C30, 90C26, 90C06, 65M55, 68T07 We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism. |
| title | Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies |
| topic | Numerical Analysis Machine Learning Optimization and Control 90C30, 90C26, 90C06, 65M55, 68T07 |
| url | https://arxiv.org/abs/2306.17648 |