EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization

Documenti analoghi

• Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
  di: Glentis, Athanasios, et al.
  Pubblicazione: (2025)
• A Stochastic Approximation Approach for Efficient Decentralized Optimization on Random Networks
  di: Yau, Chung-Yiu, et al.
  Pubblicazione: (2024)
• EMC$^2$: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence
  di: Yau, Chung-Yiu, et al.
  Pubblicazione: (2024)
• Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates
  di: Glentis, Athanasios, et al.
  Pubblicazione: (2026)
• Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
  di: Xie, Xingyu, et al.
  Pubblicazione: (2022)