Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

Documenti analoghi

• Accelerated Gradient Tracking over Time-varying Graphs for Decentralized Optimization
  di: Li, Huan, et al.
  Pubblicazione: (2021)
• EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
  di: Yau, Chung-Yiu, et al.
  Pubblicazione: (2026)
• Optimization Hyper-parameter Laws for Large Language Models
  di: Xie, Xingyu, et al.
  Pubblicazione: (2024)
• Provable Accelerated Convergence of Nesterov's Momentum for Deep ReLU Neural Networks
  di: Liao, Fangshuo, et al.
  Pubblicazione: (2023)
• On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
  di: Li, Huan, et al.
  Pubblicazione: (2025)