A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912605841391616 |
|---|---|
| author | Toscano, Juan Diego Chen, Daniel T. Oommen, Vivek Darbon, Jérôme Karniadakis, George Em |
| author_facet | Toscano, Juan Diego Chen, Daniel T. Oommen, Vivek Darbon, Jérôme Karniadakis, George Em |
| contents | Residual-based adaptive strategies are widely used in scientific machine learning but remain largely heuristic. We introduce a unifying variational framework that formalizes these methods by integrating convex transformations of the residual. Different transformations correspond to distinct objective functionals: exponential weights target the minimization of uniform error, while linear weights recover the minimization of quadratic error. Within this perspective, adaptive weighting is equivalent to selecting sampling distributions that optimize the primal objective, thereby linking discretization choices directly to error metrics. This principled approach yields three benefits: (1) it enables systematic design of adaptive schemes across norms, (2) reduces discretization error through variance reduction of the loss estimator, and (3) enhances learning dynamics by improving the gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across optimizers and architectures. Our results provide a theoretical justification of residual-based adaptivity and establish a foundation for principled discretization and training strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14198 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning Toscano, Juan Diego Chen, Daniel T. Oommen, Vivek Darbon, Jérôme Karniadakis, George Em Machine Learning Numerical Analysis Optimization and Control Computational Physics Residual-based adaptive strategies are widely used in scientific machine learning but remain largely heuristic. We introduce a unifying variational framework that formalizes these methods by integrating convex transformations of the residual. Different transformations correspond to distinct objective functionals: exponential weights target the minimization of uniform error, while linear weights recover the minimization of quadratic error. Within this perspective, adaptive weighting is equivalent to selecting sampling distributions that optimize the primal objective, thereby linking discretization choices directly to error metrics. This principled approach yields three benefits: (1) it enables systematic design of adaptive schemes across norms, (2) reduces discretization error through variance reduction of the loss estimator, and (3) enhances learning dynamics by improving the gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across optimizers and architectures. Our results provide a theoretical justification of residual-based adaptivity and establish a foundation for principled discretization and training strategies. |
| title | A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning |
| topic | Machine Learning Numerical Analysis Optimization and Control Computational Physics |
| url | https://arxiv.org/abs/2509.14198 |