What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Documenti analoghi

• Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
  di: Singh, Aaditya K., et al.
  Pubblicazione: (2025)
• Softmax $\geq$ Linear: Transformers may learn to classify in-context by kernel gradient descent
  di: Dragutinović, Sara, et al.
  Pubblicazione: (2025)
• The broader spectrum of in-context learning
  di: Lampinen, Andrew Kyle, et al.
  Pubblicazione: (2024)
• HARP: A challenging human-annotated math reasoning benchmark
  di: Yue, Albert S., et al.
  Pubblicazione: (2024)
• Training Dynamics of In-Context Learning in Linear Attention
  di: Zhang, Yedi, et al.
  Pubblicazione: (2025)