Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.09699 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909876818542592 |
|---|---|
| author | Sapienza, Facundo Bolibar, Jordi Schäfer, Frank Groenke, Brian Pal, Avik Boussange, Victor Heimbach, Patrick Hooker, Giles Pérez, Fernando Persson, Per-Olof Rackauckas, Christopher |
| author_facet | Sapienza, Facundo Bolibar, Jordi Schäfer, Frank Groenke, Brian Pal, Avik Boussange, Victor Heimbach, Patrick Hooker, Giles Pérez, Fernando Persson, Per-Olof Rackauckas, Christopher |
| contents | The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09699 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Differentiable Programming for Differential Equations: A Review Sapienza, Facundo Bolibar, Jordi Schäfer, Frank Groenke, Brian Pal, Avik Boussange, Victor Heimbach, Patrick Hooker, Giles Pérez, Fernando Persson, Per-Olof Rackauckas, Christopher Numerical Analysis Dynamical Systems Computational Physics Machine Learning 34-04, 49K40, 65D25, 65L09, 65M32, 86A22, 90C31 The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling. |
| title | Differentiable Programming for Differential Equations: A Review |
| topic | Numerical Analysis Dynamical Systems Computational Physics Machine Learning 34-04, 49K40, 65D25, 65L09, 65M32, 86A22, 90C31 |
| url | https://arxiv.org/abs/2406.09699 |