Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training

Documenti analoghi

• Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
  di: Baek, Beomhan, et al.
  Pubblicazione: (2025)
• Does SGD really happen in tiny subspaces?
  di: Song, Minhak, et al.
  Pubblicazione: (2024)
• Linear attention is (maybe) all you need (to understand transformer optimization)
  di: Ahn, Kwangjun, et al.
  Pubblicazione: (2023)
• How to escape sharp minima with random perturbations
  di: Ahn, Kwangjun, et al.
  Pubblicazione: (2023)
• Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
  di: Kim, Jihwan, et al.
  Pubblicazione: (2026)