Specialization of softmax attention heads: insights from the high-dimensional single-location model

Documenti analoghi

• Asymptotic generalization error of a single-layer graph convolutional network
  di: Duranthon, O., et al.
  Pubblicazione: (2024)
• Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block model
  di: Duranthon, O., et al.
  Pubblicazione: (2025)
• Statistical Advantage of Softmax Attention: Insights from Single-Location Regression
  di: Duranthon, O., et al.
  Pubblicazione: (2025)
• Fundamental limits of learning in sequence multi-index models and deep attention networks: High-dimensional asymptotics and sharp thresholds
  di: Troiani, Emanuele, et al.
  Pubblicazione: (2025)
• Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions
  di: Keup, Christian, et al.
  Pubblicazione: (2024)