Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers

Documenti analoghi

• Clustering and Alignment: Understanding the Training Dynamics in Modular Addition
  di: Musat, Tiberiu
  Pubblicazione: (2024)
• The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold
  di: Musat, Tiberiu
  Pubblicazione: (2025)
• Neural Weight Norm = Kolmogorov Complexity
  di: Musat, Tiberiu
  Pubblicazione: (2026)
• Improving Transformers with Dynamically Composable Multi-Head Attention
  di: Xiao, Da, et al.
  Pubblicazione: (2024)
• On the Emergence of Induction Heads for In-Context Learning
  di: Musat, Tiberiu, et al.
  Pubblicazione: (2025)