Image de couverture de livre

Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Yedi, Saxe, Andrew, Latham, Peter E.
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

When Are Bias-Free ReLU Networks Effectively Linear Networks?
par: Zhang, Yedi, et autres
Publié: (2024)

Understanding Unimodal Bias in Multimodal Deep Linear Networks
par: Zhang, Yedi, et autres
Publié: (2023)

Training Dynamics of In-Context Learning in Linear Attention
par: Zhang, Yedi, et autres
Publié: (2025)

Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape
par: Bantzis, Ioannis, et autres
Publié: (2025)

Neural Network-based High-index Saddle Dynamics Method for Searching Saddle Points and Solution Landscape
par: Liu, Yuankai, et autres
Publié: (2024)

Stochastic Gradient Descent in the Saddle-to-Saddle Regime of Deep Linear Networks
par: Corlouer, Guillaume, et autres
Publié: (2026)

Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks
par: Zhang, Leyang, et autres
Publié: (2024)

Plateaus, Optima, and Overfitting in Multi-Layer Perceptrons: A Saddle-Saddle-Attractor Scenario
par: Maleknia, Alex Alì, et autres
Publié: (2026)

Series of Hessian-Vector Products for Tractable Saddle-Free Newton Optimisation of Neural Networks
par: Oldewage, Elre T., et autres
Publié: (2023)

Directional Convergence Near Small Initializations and Saddles in Two-Homogeneous Neural Networks
par: Kumar, Akshay, et autres
Publié: (2024)

A Theory of Saddle Escape in Deep Nonlinear Networks
par: Rawal, Divit, et autres
Publié: (2026)

Regret Minimization via Saddle Point Optimization
par: Kirschner, Johannes, et autres
Publié: (2024)

Dimension-Free Saddle-Point Escape in Muon
par: Long, Yanlin, et autres
Publié: (2026)

Federated Composite Saddle Point Optimization
par: Bai, Site, et autres
Publié: (2023)

Hierarchical Simplicity Bias of Neural Networks
par: Du, Zhehang
Publié: (2023)

Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
par: Bylinkin, Dmitry, et autres
Publié: (2025)

Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding
par: Wu, Frank Zhengqing, et autres
Publié: (2024)

Saddle Networks: Structure-Preserving Architectures for Convex-Concave Functions
par: Warin, Xavier
Publié: (2026)

Saddle Hierarchy in Dense Associative Memory
par: Thériault, Robin, et autres
Publié: (2025)

Never Saddle for Reparameterized Steepest Descent as Mirror Flow
par: Jacobs, Tom, et autres
Publié: (2026)

Quantization Avoids Saddle Points in Distributed Optimization
par: Bo, Yanan, et autres
Publié: (2024)

Dimer-Enhanced Optimization: A First-Order Approach to Escaping Saddle Points in Neural Network Training
par: Hu, Yue, et autres
Publié: (2025)

Inertial Newton Algorithms Avoiding Strict Saddle Points
par: Castera, Camille
Publié: (2021)

Proximal Point Method for Online Saddle Point Problem
par: Meng, Qing-xin, et autres
Publié: (2024)

Efficiently Escaping Saddle Points for Policy Optimization
par: Khorasani, Sadegh, et autres
Publié: (2023)

A Saddle Point Remedy: Power of Variable Elimination in Non-convex Optimization
par: Gan, Min, et autres
Publié: (2025)

Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?
par: Innocenti, Francesco, et autres
Publié: (2024)

Simultaneous Learning and Optimization via Misspecified Saddle Point Problems
par: Ahmadi, Mohammad Mahdi, et autres
Publié: (2025)

Do Quantum Neural Networks have Simplicity Bias?
par: Pointing, Jessica
Publié: (2024)

Optimal Learning Rate Schedule for Balancing Effort and Performance
par: Njaradi, Valentina, et autres
Publié: (2026)

Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points
par: Yamamoto, Naoya, et autres
Publié: (2025)

Algorithm Development in Neural Networks: Insights from the Streaming Parity Task
par: van Rossem, Loek, et autres
Publié: (2025)

Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
par: Beznosikov, Aleksandr, et autres
Publié: (2026)

Online Min-Max Optimization: From Individual Regrets to Cumulative Saddle Points
par: Vyas, Abhijeet, et autres
Publié: (2026)

Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent
par: Ziyin, Liu, et autres
Publié: (2023)

Bilevel Optimization over Saddle Points of Zero-Sum Markov Games
par: Zheng, Zihao, et autres
Publié: (2026)

A Compression Perspective on Simplicity Bias
par: Marty, Tom, et autres
Publié: (2026)

WinQ: Accelerating Quantization-Aware Training of Language Models Around Saddle Points
par: Li, Dongyue, et autres
Publié: (2026)

Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
par: Cao, Daniel Yiming, et autres
Publié: (2025)

Escaping Saddle Points for Nonsmooth Weakly Convex Functions via Perturbed Proximal Algorithms
par: Huang, Minhui, et autres
Publié: (2021)