How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression

Documents similaires

• What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
  par: Chen, Xingwu, et autres
  Publié: (2024)
• Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
  par: Chen, Xingwu, et autres
  Publié: (2025)
• On the Robustness of Transformers against Context Hijacking for Linear Classification
  par: Li, Tianle, et autres
  Publié: (2025)
• How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
  par: Wu, Jingfeng, et autres
  Publié: (2023)
• Superiority of Multi-Head Attention in In-Context Linear Regression
  par: Cui, Yingqian, et autres
  Publié: (2024)