Optimal Learning-Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay

Documenti analoghi

• Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules
  di: Li, Binghui, et al.
  Pubblicazione: (2025)
• Training Dynamics of the Cooldown Stage in Warmup-Stable-Decay Learning Rate Scheduler
  di: Dremov, Aleksandr, et al.
  Pubblicazione: (2025)
• Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
  di: Wen, Kaiyue, et al.
  Pubblicazione: (2024)
• Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers
  di: Belloni, Annalisa, et al.
  Pubblicazione: (2026)
• Optimal Linear Decay Learning Rate Schedules and Further Refinements
  di: Defazio, Aaron, et al.
  Pubblicazione: (2023)