Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers

Documenti analoghi

• Learning Linear Regression with Low-Rank Tasks in-Context
  di: Takanami, Kaito, et al.
  Pubblicazione: (2025)
• Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
  di: Atanasov, Alexander, et al.
  Pubblicazione: (2025)
• Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theory
  di: Nishiyama, Sota, et al.
  Pubblicazione: (2025)
• Infinite Limits of Multi-head Transformer Dynamics
  di: Bordelon, Blake, et al.
  Pubblicazione: (2024)
• Geometric Dynamics of Signal Propagation Predict Trainability of Transformers
  di: Cowsik, Aditya, et al.
  Pubblicazione: (2024)