How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data

Documenti analoghi

• Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
  di: Vasudeva, Bhavya, et al.
  Pubblicazione: (2026)
• Implicit Bias and Fast Convergence Rates for Self-attention
  di: Vasudeva, Bhavya, et al.
  Pubblicazione: (2024)
• In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
  di: Deora, Puneesh, et al.
  Pubblicazione: (2025)
• Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization
  di: Behnia, Tina, et al.
  Pubblicazione: (2025)
• On the Optimization and Generalization of Multi-head Attention
  di: Deora, Puneesh, et al.
  Pubblicazione: (2023)