Benignity of loss landscape with weight decay requires both large overparametrization and initialization

Documenti analoghi

• Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias
  di: Town, James, et al.
  Pubblicazione: (2026)
• First-order ANIL provably learns representations despite overparametrization
  di: Yüksel, Oğuz Kaan, et al.
  Pubblicazione: (2023)
• Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs
  di: Boursier, Etienne, et al.
  Pubblicazione: (2022)
• Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective
  di: Boursier, Etienne, et al.
  Pubblicazione: (2025)
• Early alignment in two-layer networks training is a two-edged sword
  di: Boursier, Etienne, et al.
  Pubblicazione: (2024)