Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.13644 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914169049055232 |
|---|---|
| author | Dangel, Felix Siebert, Tim Zeinhofer, Marius Walther, Andrea |
| author_facet | Dangel, Felix Siebert, Tim Zeinhofer, Marius Walther, Andrea |
| contents | Computing partial differential equation (PDE) operators via nested backpropagation is expensive, yet popular, and severely restricts their utility for scientific machine learning. Recent advances, like the forward Laplacian and randomizing Taylor mode automatic differentiation (AD), propose forward schemes to address this. We introduce an optimization technique for Taylor mode that 'collapses' derivatives by rewriting the computational graph, and demonstrate how to apply it to general linear PDE operators, and randomized Taylor mode. The modifications simply require propagating a sum up the computational graph, which could -- or should -- be done by a machine learning compiler, without exposing complexity to users. We implement our collapsing procedure and evaluate it on popular PDE operators, confirming it accelerates Taylor mode and outperforms nested backpropagation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13644 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Collapsing Taylor Mode Automatic Differentiation Dangel, Felix Siebert, Tim Zeinhofer, Marius Walther, Andrea Machine Learning Computing partial differential equation (PDE) operators via nested backpropagation is expensive, yet popular, and severely restricts their utility for scientific machine learning. Recent advances, like the forward Laplacian and randomizing Taylor mode automatic differentiation (AD), propose forward schemes to address this. We introduce an optimization technique for Taylor mode that 'collapses' derivatives by rewriting the computational graph, and demonstrate how to apply it to general linear PDE operators, and randomized Taylor mode. The modifications simply require propagating a sum up the computational graph, which could -- or should -- be done by a machine learning compiler, without exposing complexity to users. We implement our collapsing procedure and evaluate it on popular PDE operators, confirming it accelerates Taylor mode and outperforms nested backpropagation. |
| title | Collapsing Taylor Mode Automatic Differentiation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.13644 |