Cover Image

$ε$-rank and the Staircase Phenomenon: New Insights into Neural Network Training Dynamics

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Jiang, Zhao, Yuxiang, Zhu, Quanhui
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Numerical Analysis
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Structured First-Layer Initialization Pre-Training Techniques to Accelerate Training Process Based on $\varepsilon$-Rank
by: Tang, Tao, et al.
Published: (2025)

Energy Dissipation Preserving Feature-based DNN Galerkin Methods for Gradient Flows
by: Tang, Tao, et al.
Published: (2026)

Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations
by: Chen, Chuqi, et al.
Published: (2024)

Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers
by: Chen, Chuqi, et al.
Published: (2024)

Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
by: Schotthöfer, Steffen, et al.
Published: (2025)

A New Tensor Network: Tubal Tensor Train and Its Applications
by: Ahmadi-Asl, Salman, et al.
Published: (2026)

Deep Neural Network Solutions for Oscillatory Fredholm Integral Equations
by: Jiang, Jie, et al.
Published: (2024)

Data-Parallel Neural Network Training via Nonlinearly Preconditioned Trust-Region Method
by: Alegría, Samuel A. Cruz, et al.
Published: (2025)

ZNO: Stable Rational Neural Operators in the Z-Domain for Discrete-Time Dynamics
by: Zhu, Xianli, et al.
Published: (2026)

Neural Network Approach to Stochastic Dynamics for Smooth Multimodal Density Estimation
by: Zarezadeh, Z., et al.
Published: (2025)

Preconditioning for Physics-Informed Neural Networks
by: Liu, Songming, et al.
Published: (2024)

A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks
by: Sun, Yuxin, et al.
Published: (2022)

Efficient Differentiable Approximation of Generalized Low-rank Regularization
by: Li, Naiqi, et al.
Published: (2025)

Dual-Balancing for Physics-Informed Neural Networks
by: Zhou, Chenhong, et al.
Published: (2025)

Why Cannot Neural Networks Master Extrapolation? Insights from Physical Laws
by: Dakhmouche, Ramzi, et al.
Published: (2025)

A Multiple Transferable Neural Network Method with Domain Decomposition for Elliptic Interface Problems
by: Lu, Tianzheng, et al.
Published: (2025)

Physics-embedded Fourier Neural Network for Partial Differential Equations
by: Xu, Qingsong, et al.
Published: (2024)

Multi-Level Monte Carlo Training of Neural Operators
by: Rowbottom, James, et al.
Published: (2025)

On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functions
by: Lu, Yulong, et al.
Published: (2025)

Multigrade Neural Network Approximation
by: Zhang, Shijun, et al.
Published: (2026)

Adaptive-Distribution Randomized Neural Networks for PDEs: A Low-Dimensional Distribution-Learning Framework
by: Yang, You, et al.
Published: (2026)

DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving
by: Yue, Xihang, et al.
Published: (2024)

TINNs: Time-Induced Neural Networks for Solving Time-Dependent PDEs
by: Dai, Chen-Yang, et al.
Published: (2026)

THINNs: Thermodynamically Informed Neural Networks
by: Castro, Javier, et al.
Published: (2025)

Generative Feature Training of Thin 2-Layer Networks
by: Hertrich, Johannes, et al.
Published: (2024)

Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery
by: Liu, Ning, et al.
Published: (2025)

Mixture-of-Experts Operator Transformer for Large-Scale PDE Pre-Training
by: Wang, Hong, et al.
Published: (2025)

Dual Cone Gradient Descent for Training Physics-Informed Neural Networks
by: Hwang, Youngsik, et al.
Published: (2024)

Solving Roughly Forced Nonlinear PDEs via Misspecified Kernel Methods and Neural Networks
by: Baptista, Ricardo, et al.
Published: (2025)

Estimating condition number with Graph Neural Networks
by: Carson, Erin, et al.
Published: (2026)

Guaranteed Sampling Flexibility for Low-tubal-rank Tensor Completion
by: Su, Bowen, et al.
Published: (2024)

A randomized algorithm to solve reduced rank operator regression
by: Turri, Giacomo, et al.
Published: (2023)

An Augmented Backward-Corrected Projector Splitting Integrator for Dynamical Low-Rank Training
by: Kusch, Jonas, et al.
Published: (2025)

Decentralized Neural Networks for Robust and Scalable Eigenvalue Computation
by: Katende, Ronald
Published: (2024)

Parallel-in-Time Solutions with Random Projection Neural Networks
by: Betcke, Marta M., et al.
Published: (2024)

AutoBalance: An Automatic Balancing Framework for Training Physics-Informed Neural Networks
by: An, Kang, et al.
Published: (2025)

E-PINNs: Epistemic Physics-Informed Neural Networks
by: Jacob, Bruno, et al.
Published: (2025)

Deep NURBS -- Admissible Physics-informed Neural Networks
by: Saidaoui, Hamed, et al.
Published: (2022)

From Simple to Complex: Curriculum-Guided Physics-Informed Neural Networks via Gaussian Mixture Models
by: Yang, Jianan, et al.
Published: (2026)

U-HNO: A U-shaped Hybrid Neural Operator with Sparse-Point Adaptive Routing for Non-stationary PDE Dynamics
by: Ma, Yingzhe, et al.
Published: (2026)