Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape

Documenti analoghi

• Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding
  di: Wu, Frank Zhengqing, et al.
  Pubblicazione: (2024)
• Dimer-Enhanced Optimization: A First-Order Approach to Escaping Saddle Points in Neural Network Training
  di: Hu, Yue, et al.
  Pubblicazione: (2025)
• Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures
  di: Zhang, Yedi, et al.
  Pubblicazione: (2025)
• Stochastic Gradient Descent in the Saddle-to-Saddle Regime of Deep Linear Networks
  di: Corlouer, Guillaume, et al.
  Pubblicazione: (2026)
• A Theory of Saddle Escape in Deep Nonlinear Networks
  di: Rawal, Divit, et al.
  Pubblicazione: (2026)