Cover Image

Numerical Fragility in Transformers: A Layer-wise Theory for Explaining, Forecasting, and Mitigating Instability

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Baek, Jinwoo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Numerical Analysis
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Matrix Phylogeny: Compact Spectral Fingerprints for Trap-Robust Preconditioner Selection
by: Baek, Jinwoo
Published: (2025)

Chebyshev Moment Regularization (CMR): Condition-Number Control with Moment Shaping
by: Baek, Jinwoo
Published: (2025)

Geometric Layer-wise Approximation Rates for Deep Networks
by: Zhang, Shijun, et al.
Published: (2026)

Power Transform Revisited: Numerically Stable, and Federated
by: Xu, Xuefeng, et al.
Published: (2025)

Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
by: Islam, Chashi Mahiul, et al.
Published: (2026)

N-Adaptive Ritz Method: A Neural Network Enriched Partition of Unity for Boundary Value Problems
by: Baek, Jonghyuk, et al.
Published: (2024)

The Numerical Stability of Hyperbolic Representation Learning
by: Mishne, Gal, et al.
Published: (2022)

Numerical Error Analysis of Large Language Models
by: Budzinskiy, Stanislav, et al.
Published: (2025)

Artifacts of Numerical Integration in Learning Dynamical Systems
by: Lu, Bing-Ze, et al.
Published: (2025)

Parallel-in-Time Probabilistic Numerical ODE Solvers
by: Bosch, Nathanael, et al.
Published: (2023)

Numerical Considerations for the Construction of Karhunen-Loève Expansions
by: Safta, Cosmin, et al.
Published: (2026)

A Hybridizable Neural Time Integrator for Stable Autoregressive Forecasting
by: Kinch, Brooks, et al.
Published: (2026)

Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networks
by: Qian, Yianxia, et al.
Published: (2024)

Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers
by: Zhang, Enrui, et al.
Published: (2022)

A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks
by: Sun, Yuxin, et al.
Published: (2022)

Pychop: Emulating Low-Precision Arithmetic in Numerical Methods and Neural Networks
by: Carson, Erin, et al.
Published: (2025)

Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebra
by: Bioli, Ivan, et al.
Published: (2025)

Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning
by: Dereziński, Michał, et al.
Published: (2024)

Propagating Model Uncertainty through Filtering-based Probabilistic Numerical ODE Solvers
by: Yao, Dingling, et al.
Published: (2025)

Fuzzy PyTorch: Rapid Numerical Variability Evaluation for Deep Learning Models
by: Gonzalez-Pepe, Inés, et al.
Published: (2026)

Approximating Numerical Fluxes Using Fourier Neural Operators for Hyperbolic Conservation Laws
by: Kim, Taeyoung, et al.
Published: (2024)

Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving
by: Arisaka, Sohei, et al.
Published: (2024)

Numerical Analysis of HiPPO-LegS ODE for Deep State Space Models
by: Park, Jaesung R., et al.
Published: (2024)

Singularity Formation: Synergy in Theoretical, Numerical and Machine Learning Approaches
by: Wang, Yixuan
Published: (2026)

Efficient Numerical Integration in Reproducing Kernel Hilbert Spaces via Leverage Scores Sampling
by: Chatalic, Antoine, et al.
Published: (2023)

A Numerical Method for Coupling Parameterized Physics-Informed Neural Networks and FDM for Advanced Thermal-Hydraulic System Simulation
by: Shin, Jeesuk, et al.
Published: (2026)

Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarsening
by: Li, Guoming, et al.
Published: (2025)

Generative Feature Training of Thin 2-Layer Networks
by: Hertrich, Johannes, et al.
Published: (2024)

Ensemble Score Filtering for Real-Data Energy Consumption Forecast Correction
by: Hu, Ruoyu, et al.
Published: (2026)

A Computationally Efficient Multidimensional Vision Transformer
by: Ichi, Alaa El, et al.
Published: (2026)

Numerical simulation of transient heat conduction with moving heat source using Physics Informed Neural Networks
by: Kalyan, Anirudh, et al.
Published: (2025)

Inverse Evolution Layers: Physics-informed Regularizers for Deep Neural Networks
by: Liu, Chaoyu, et al.
Published: (2023)

ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance
by: Zheng, Haolan, et al.
Published: (2025)

Machine Learning Based Optimization Workflow for Tuning Numerical Settings of Differential Equation Solvers for Boundary Value Problems
by: Victor, Viny Saajan, et al.
Published: (2024)

Transolver: A Fast Transformer Solver for PDEs on General Geometries
by: Wu, Haixu, et al.
Published: (2024)

Training-Free Looped Transformers
by: Chen, Lizhang, et al.
Published: (2026)

A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theory
by: Weihs, Adrien, et al.
Published: (2025)

A hybrid iterative method based on MIONet for PDEs: Theory and numerical examples
by: Hu, Jun, et al.
Published: (2024)

Sketch 'n Solve: An Efficient Python Package for Large-Scale Least Squares Using Randomized Numerical Linear Algebra
by: Lavaee, Alex
Published: (2024)

Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks
by: Jin, Bangti, et al.
Published: (2025)