On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization

Documenti analoghi

• Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
  di: Sahu, Sharan, et al.
  Pubblicazione: (2026)
• Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
  di: Vaswani, Sharan, et al.
  Pubblicazione: (2025)
• (Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
  di: Dang, Anh, et al.
  Pubblicazione: (2024)
• $μ^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism
  di: Dahan, Tehila, et al.
  Pubblicazione: (2023)
• Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
  di: Jiang, Ruichen, et al.
  Pubblicazione: (2024)