Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910393866125312 |
|---|---|
| author | Zhang, Huan Chen, Yifan Vanden-Eijnden, Eric Peherstorfer, Benjamin |
| author_facet | Zhang, Huan Chen, Yifan Vanden-Eijnden, Eric Peherstorfer, Benjamin |
| contents | Sequential-in-time methods solve a sequence of training problems to fit nonlinear parametrizations such as neural networks to approximate solution trajectories of partial differential equations over time. This work shows that sequential-in-time training methods can be understood broadly as either optimize-then-discretize (OtD) or discretize-then-optimize (DtO) schemes, which are well known concepts in numerical analysis. The unifying perspective leads to novel stability and a posteriori error analysis results that provide insights into theoretical and numerical aspects that are inherent to either OtD or DtO schemes such as the tangent space collapse phenomenon, which is a form of over-fitting. Additionally, the unified perspective facilitates establishing connections between variants of sequential-in-time training methods, which is demonstrated by identifying natural gradient descent methods on energy functionals as OtD schemes applied to the corresponding gradient flows. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01145 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations Zhang, Huan Chen, Yifan Vanden-Eijnden, Eric Peherstorfer, Benjamin Numerical Analysis Machine Learning Sequential-in-time methods solve a sequence of training problems to fit nonlinear parametrizations such as neural networks to approximate solution trajectories of partial differential equations over time. This work shows that sequential-in-time training methods can be understood broadly as either optimize-then-discretize (OtD) or discretize-then-optimize (DtO) schemes, which are well known concepts in numerical analysis. The unifying perspective leads to novel stability and a posteriori error analysis results that provide insights into theoretical and numerical aspects that are inherent to either OtD or DtO schemes such as the tangent space collapse phenomenon, which is a form of over-fitting. Additionally, the unified perspective facilitates establishing connections between variants of sequential-in-time training methods, which is demonstrated by identifying natural gradient descent methods on energy functionals as OtD schemes applied to the corresponding gradient flows. |
| title | Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations |
| topic | Numerical Analysis Machine Learning |
| url | https://arxiv.org/abs/2404.01145 |