Optimal Attention Temperature Improves the Robustness of In-Context Learning under Distribution Shift in High Dimensions

Documenti analoghi

• How Data Mixing Shapes In-Context Learning: Asymptotic Equivalence for Transformers with MLPs
  di: Demir, Samet, et al.
  Pubblicazione: (2025)
• Input-Label Correlation Governs a Linear-to-Nonlinear Transition in Random Features under Spiked Covariance
  di: Demir, Samet, et al.
  Pubblicazione: (2024)
• Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure
  di: Demir, Samet, et al.
  Pubblicazione: (2025)
• Implicitly Normalized Online PCA: A Regularized Algorithm with Exact High-Dimensional Dynamics
  di: Demir, Samet, et al.
  Pubblicazione: (2025)
• Asymptotic Study of In-context Learning with Random Transformers through Equivalent Models
  di: Demir, Samet, et al.
  Pubblicazione: (2025)