Cover Image

A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sun, Yuxin, Lao, Dong, Sundaramoorthi, Ganesh, Yezzi, Anthony
Format:	Preprint
Published:	2022
Subjects:	Machine Learning Numerical Analysis
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers
by: Zhang, Enrui, et al.
Published: (2022)

Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers
by: Chen, Chuqi, et al.
Published: (2024)

Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networks
by: Qian, Yianxia, et al.
Published: (2024)

U-HNO: A U-shaped Hybrid Neural Operator with Sparse-Point Adaptive Routing for Non-stationary PDE Dynamics
by: Ma, Yingzhe, et al.
Published: (2026)

Mixture-of-Experts Operator Transformer for Large-Scale PDE Pre-Training
by: Wang, Hong, et al.
Published: (2025)

Graph Neural Regularizers for PDE Inverse Problems
by: Lauga, William, et al.
Published: (2025)

AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing
by: Du, Jianda, et al.
Published: (2026)

Training-Free Looped Transformers
by: Chen, Lizhang, et al.
Published: (2026)

Latent Neural Operator for Solving Forward and Inverse PDE Problems
by: Wang, Tian, et al.
Published: (2024)

Neural Parameter Regression for Explicit Representations of PDE Solution Operators
by: Mundinger, Konrad, et al.
Published: (2024)

The Numerical Stability of Hyperbolic Representation Learning
by: Mishne, Gal, et al.
Published: (2022)

Pychop: Emulating Low-Precision Arithmetic in Numerical Methods and Neural Networks
by: Carson, Erin, et al.
Published: (2025)

Learning Neural PDE Solvers with Convergence Guarantees
by: Hsieh, Jun-Ting, et al.
Published: (2019)

Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers
by: Zhou, Hang, et al.
Published: (2024)

DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
by: Hao, Zhongkai, et al.
Published: (2024)

BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics
by: Liu, Yuxuan, et al.
Published: (2025)

Moving Sampling Physics-informed Neural Networks induced by Moving Mesh PDE
by: Yang, Yu, et al.
Published: (2023)

Training with Fewer Bits: Unlocking Edge LLMs Training with Stochastic Rounding
by: Liu, Taowen, et al.
Published: (2025)

Are Deep Learning Based Hybrid PDE Solvers Reliable? Why Training Paradigms and Update Strategies Matter
by: Wu, Yuhan, et al.
Published: (2026)

Stabilizing PDE--ML coupled systems
by: Qadeer, Saad, et al.
Published: (2025)

DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving
by: Yue, Xihang, et al.
Published: (2024)

Dual Cone Gradient Descent for Training Physics-Informed Neural Networks
by: Hwang, Youngsik, et al.
Published: (2024)

PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencils
by: Cheng, Chun-Wun, et al.
Published: (2025)

Numerical simulation of transient heat conduction with moving heat source using Physics Informed Neural Networks
by: Kalyan, Anirudh, et al.
Published: (2025)

A Numerical Method for Coupling Parameterized Physics-Informed Neural Networks and FDM for Advanced Thermal-Hydraulic System Simulation
by: Shin, Jeesuk, et al.
Published: (2026)

CATO: Charted Attention for Neural PDE Operators
by: Cheng, Chun-Wun, et al.
Published: (2026)

Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations
by: Chen, Chuqi, et al.
Published: (2024)

$ε$-rank and the Staircase Phenomenon: New Insights into Neural Network Training Dynamics
by: Yang, Jiang, et al.
Published: (2024)

Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
by: Zhang, Rui, et al.
Published: (2023)

Data-Parallel Neural Network Training via Nonlinearly Preconditioned Trust-Region Method
by: Alegría, Samuel A. Cruz, et al.
Published: (2025)

A Microstructure-based Graph Neural Network for Accelerating Multiscale Simulations
by: Storm, J., et al.
Published: (2024)

PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling
by: Jo, Minju, et al.
Published: (2025)

Randomized Neural Networks for Partial Differential Equation on Static and Evolving Surfaces
by: Sun, Jingbo, et al.
Published: (2026)

A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions
by: Negrini, Elisa, et al.
Published: (2025)

Mathematics of Digital Twins and Transfer Learning for PDE Models
by: Zong, Yifei, et al.
Published: (2025)

Lattice-based Deep Neural Networks: Regularity and Tailored Regularization
by: Keller, Alexander, et al.
Published: (2026)

Approximating Numerical Fluxes Using Fourier Neural Operators for Hyperbolic Conservation Laws
by: Kim, Taeyoung, et al.
Published: (2024)

Escaping Spectral Bias without Backpropagation: Fast Implicit Neural Representations with Extreme Learning Machines
by: Cho, Woojin, et al.
Published: (2026)

Better Neural PDE Solvers Through Data-Free Mesh Movers
by: Hu, Peiyan, et al.
Published: (2023)

Solving Inverse Parametrized Problems via Finite Elements and Extreme Learning Networks
by: Burman, Erik, et al.
Published: (2026)