Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis

Documenti analoghi

• When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
  di: Li, Hongkang, et al.
  Pubblicazione: (2025)
• How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance
  di: Li, Hongkang, et al.
  Pubblicazione: (2024)
• What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding
  di: Li, Hongkang, et al.
  Pubblicazione: (2024)
• How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?
  di: Li, Hongkang, et al.
  Pubblicazione: (2024)
• Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
  di: Li, Hongkang, et al.
  Pubblicazione: (2024)