In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention

Documenti analoghi

• Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
  di: Chen, Siyu, et al.
  Pubblicazione: (2024)
• Superiority of Multi-Head Attention in In-Context Linear Regression
  di: Cui, Yingqian, et al.
  Pubblicazione: (2024)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
  di: He, Jianliang, et al.
  Pubblicazione: (2026)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
  di: He, Jianliang, et al.
  Pubblicazione: (2024)
• Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
  di: Chen, Siyu, et al.
  Pubblicazione: (2024)