In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention

Ähnliche Einträge

• Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
  von: Chen, Siyu, et al.
  Veröffentlicht: (2024)
• Superiority of Multi-Head Attention in In-Context Linear Regression
  von: Cui, Yingqian, et al.
  Veröffentlicht: (2024)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
  von: He, Jianliang, et al.
  Veröffentlicht: (2026)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
  von: He, Jianliang, et al.
  Veröffentlicht: (2024)
• Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs
  von: Guo, Tianyu, et al.
  Veröffentlicht: (2024)