Image de couverture de livre

A Theory of Saddle Escape in Deep Nonlinear Networks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Rawal, Divit, DeWeese, Michael R.
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Disordered Systems and Neural Networks
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Saddle Hierarchy in Dense Associative Memory
par: Thériault, Robin, et autres
Publié: (2025)

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
par: Lauditi, Clarissa, et autres
Publié: (2026)

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
par: Dandi, Yatin, et autres
Publié: (2026)

Dynamical Mean-Field Theory of Self-Attention Neural Networks
par: Poc-López, Ángel, et autres
Publié: (2024)

Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theory
par: Nishiyama, Sota, et autres
Publié: (2025)

The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
par: Mao, Jialin, et autres
Publié: (2023)

From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning
par: Rubin, Noa, et autres
Publié: (2025)

Kernel Renormalization in Bayesian Deep Neural Networks: the Equivalent Wishart Ansatz in the Proportional Regime
par: Baglioni, Paolo, et autres
Publié: (2026)

How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model
par: Tomasini, Umberto, et autres
Publié: (2024)

Statistical Physics of Deep Neural Networks: Generalization Capability, Beyond the Infinite Width, and Feature Learning
par: Ariosto, Sebastiano
Publié: (2025)

Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
par: Bordelon, Blake, et autres
Publié: (2025)

Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit
par: Meng, Lineghuan, et autres
Publié: (2024)

Graph Neural Networks Do Not Always Oversmooth
par: Epping, Bastian, et autres
Publié: (2024)

A Fast Algorithm to Simulate Nonlinear Resistive Networks
par: Scellier, Benjamin
Publié: (2024)

Theory of Speciation Transitions in Diffusion Models with General Class Structure
par: Achilli, Beatrice, et autres
Publié: (2026)

Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
par: Bordelon, Blake, et autres
Publié: (2025)

Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
par: Bocchi, Dario, et autres
Publié: (2026)

Deep neural networks from the perspective of ergodic theory
par: Zhang, Fan
Publié: (2023)

Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Model
par: Bordelon, Blake, et autres
Publié: (2026)

High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory
par: Nishiyama, Sota, et autres
Publié: (2026)

Applications of Statistical Field Theory in Deep Learning
par: Ringel, Zohar, et autres
Publié: (2025)

A Random-Matrix Criterion for Initializing Gated Recurrent Neural Networks
par: Fioratti, Tommaso, et autres
Publié: (2026)

A Federated Many-to-One Hopfield model for associative Neural Networks
par: Alessandrelli, Andrea, et autres
Publié: (2026)

Formation of Representations in Neural Networks
par: Ziyin, Liu, et autres
Publié: (2024)

Parameter Symmetry Potentially Unifies Deep Learning Theory
par: Ziyin, Liu, et autres
Publié: (2025)

Convergence Acceleration of Markov Chain Monte Carlo-based Gradient Descent by Deep Unfolding
par: Hagiwara, Ryo, et autres
Publié: (2024)

The Physics of Data and Tasks: Theories of Locality and Compositionality in Deep Learning
par: Favero, Alessandro
Publié: (2025)

Transfer Learning in Infinite Width Feature Learning Networks
par: Lauditi, Clarissa, et autres
Publié: (2025)

The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks
par: Farné, Gabriele, et autres
Publié: (2026)

Demolition and Reinforcement of Memories in Spin-Glass-like Neural Networks
par: Ventura, Enrico
Publié: (2024)

Growing Neural Networks: Dynamic Evolution through Gradient Descent
par: Radhakrishnan, Anil, et autres
Publié: (2025)

Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks
par: Montanari, Andrea, et autres
Publié: (2025)

Benchmarking Graph Neural Networks in Solving Hard Constraint Satisfaction Problems
par: Skenderi, Geri, et autres
Publié: (2026)

Controlled Langevin Dynamics for Sampling of Feedforward Neural Networks Trained with Minibatches
par: Zambon, Alessandro, et autres
Publié: (2026)

Statistical Mechanics Calculations Using Variational Autoregressive Networks and Quantum Annealing
par: Tamura, Yuta, et autres
Publié: (2024)

Grokking as a First Order Phase Transition in Two Layer Networks
par: Rubin, Noa, et autres
Publié: (2023)

Initial Guessing Bias: How Untrained Networks Favor Some Classes
par: Francazi, Emanuele, et autres
Publié: (2023)

Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks
par: Arnaboldi, Luca, et autres
Publié: (2025)

Graph Neural Network Approach to Predicting Magnetization in Quasi-One-Dimensional Ising Systems
par: Slavin, V., et autres
Publié: (2025)

Siamese Neural Network for Label-Efficient Critical Phenomena Prediction in 3D Percolation Models
par: Wang, Shanshan, et autres
Publié: (2025)