Implicit Regularization of Gradient Flow on One-Layer Softmax Attention

Documenti analoghi

• Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
  di: Chen, Siyu, et al.
  Pubblicazione: (2024)
• Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
  di: Chen, Siyu, et al.
  Pubblicazione: (2024)
• One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
  di: Li, Zihao, et al.
  Pubblicazione: (2024)
• Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders
  di: Chen, Siyu, et al.
  Pubblicazione: (2025)
• Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability
  di: Regis, Eric, et al.
  Pubblicazione: (2026)