Implicit Bias and Fast Convergence Rates for Self-attention

Documenti analoghi

• On the Optimization and Generalization of Multi-head Attention
  di: Deora, Puneesh, et al.
  Pubblicazione: (2023)
• In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
  di: Deora, Puneesh, et al.
  Pubblicazione: (2025)
• How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data
  di: Vasudeva, Bhavya, et al.
  Pubblicazione: (2025)
• Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
  di: Vasudeva, Bhavya, et al.
  Pubblicazione: (2026)
• The Rich and the Simple: On the Implicit Bias of Adam and SGD
  di: Vasudeva, Bhavya, et al.
  Pubblicazione: (2025)