Softmax $\geq$ Linear: Transformers may learn to classify in-context by kernel gradient descent

Documenti analoghi

• Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
  di: Singh, Aaditya K., et al.
  Pubblicazione: (2025)
• What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
  di: Singh, Aaditya K., et al.
  Pubblicazione: (2024)
• On the rate of convergence of an over-parametrized Transformer classifier learned by gradient descent
  di: Kohler, Michael, et al.
  Pubblicazione: (2023)
• Training Dynamics of In-Context Learning in Linear Attention
  di: Zhang, Yedi, et al.
  Pubblicazione: (2025)
• The broader spectrum of in-context learning
  di: Lampinen, Andrew Kyle, et al.
  Pubblicazione: (2024)