Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917542534053888 |
|---|---|
| author | Wang, Yuxin Hu, Yuanzhe Zhong, Xiaokun Wang, Xiaopeng Lu, Haiquan Pang, Tianyu Mahoney, Michael W. Yan, Yujun Ren, Pu Yang, Yaoqing |
| author_facet | Wang, Yuxin Hu, Yuanzhe Zhong, Xiaokun Wang, Xiaopeng Lu, Haiquan Pang, Tianyu Mahoney, Michael W. Yan, Yujun Ren, Pu Yang, Yaoqing |
| contents | Neural networks trained under different hyperparameter settings can fall into distinct training "regimes," with consistent behavior within regimes and qualitative differences across regimes. In this paper, we study such multi-regime behavior in scientific machine learning (SciML) models through a regime-aware diagnostic framework that jointly analyzes performance, training dynamics, and loss-landscape geometry. We identify three key findings: (i) a consistent three-regime structure emerges across many standard SciML models, different constraint enforcements, and various optimizer designs; (ii) optimization effectiveness is regime-specific, with no single method performing well across all regimes; and (iii) SciML models can exhibit fine-grained failure modes that can challenge conventional interpretations of standard loss-landscape metrics. Our results provide an approach to establish a unified, task-oblivious perspective on failure modes in SciML and to inform regime-aware guidance for improving robustness. We validate these findings across widely-used SciML models, including physics-informed neural networks, neural operators, and neural ordinary differential equations, on benchmarks spanning representative ordinary and partial differential equations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_29153 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization Wang, Yuxin Hu, Yuanzhe Zhong, Xiaokun Wang, Xiaopeng Lu, Haiquan Pang, Tianyu Mahoney, Michael W. Yan, Yujun Ren, Pu Yang, Yaoqing Machine Learning Artificial Intelligence Computational Physics Neural networks trained under different hyperparameter settings can fall into distinct training "regimes," with consistent behavior within regimes and qualitative differences across regimes. In this paper, we study such multi-regime behavior in scientific machine learning (SciML) models through a regime-aware diagnostic framework that jointly analyzes performance, training dynamics, and loss-landscape geometry. We identify three key findings: (i) a consistent three-regime structure emerges across many standard SciML models, different constraint enforcements, and various optimizer designs; (ii) optimization effectiveness is regime-specific, with no single method performing well across all regimes; and (iii) SciML models can exhibit fine-grained failure modes that can challenge conventional interpretations of standard loss-landscape metrics. Our results provide an approach to establish a unified, task-oblivious perspective on failure modes in SciML and to inform regime-aware guidance for improving robustness. We validate these findings across widely-used SciML models, including physics-informed neural networks, neural operators, and neural ordinary differential equations, on benchmarks spanning representative ordinary and partial differential equations. |
| title | Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization |
| topic | Machine Learning Artificial Intelligence Computational Physics |
| url | https://arxiv.org/abs/2605.29153 |