How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression

Documenti analoghi

• What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
  di: Chen, Xingwu, et al.
  Pubblicazione: (2024)
• Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
  di: Chen, Xingwu, et al.
  Pubblicazione: (2025)
• On the Robustness of Transformers against Context Hijacking for Linear Classification
  di: Li, Tianle, et al.
  Pubblicazione: (2025)
• How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
  di: Wu, Jingfeng, et al.
  Pubblicazione: (2023)
• Superiority of Multi-Head Attention in In-Context Linear Regression
  di: Cui, Yingqian, et al.
  Pubblicazione: (2024)