Image de couverture de livre

Looped Transformers with Layer Normalization Provably Learn the Power Method

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Lyumin, Zhang, Chenyang, Cao, Yuan
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Transformers Efficiently Perform In-Context Logistic Regression via Normalized Gradient Descent
par: Zhang, Chenyang, et autres
Publié: (2026)

Transformers Trained via Gradient Descent Can Provably Learn a Class of Teacher Models
par: Zhang, Chenyang, et autres
Publié: (2026)

Transformer Learns Optimal Variable Selection in Group-Sparse Classification
par: Zhang, Chenyang, et autres
Publié: (2025)

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
par: Li, Zihao, et autres
Publié: (2024)

On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning
par: Zhang, Thomas T., et autres
Publié: (2025)

Analogies between Transformer Layers and Power Method
par: Li, Chenglong, et autres
Publié: (2026)

Towards Understanding Generalization in DP-GD: A Case Study in Training Two-Layer CNNs
par: Shi, Zhongjie, et autres
Publié: (2025)

Provable Knowledge Acquisition and Extraction in One-Layer Transformers
par: Xu, Ruichen, et autres
Publié: (2025)

Towards Simple and Provable Parameter-Free Adaptive Gradient Methods
par: Tao, Yuanzhe, et autres
Publié: (2024)

Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs
par: Scholkemper, Michael, et autres
Publié: (2024)

Learning Spectral Methods by Transformers
par: He, Yihan, et autres
Publié: (2025)

Transformers Provably Learn to Internalize Chain-of-Thought
par: Huang, Yixiao, et autres
Publié: (2026)

Provable In-Context Learning of Nonlinear Regression with Transformers
par: Li, Hongbo, et autres
Publié: (2025)

Stability of Transformers under Layer Normalization
par: Kan, Kelvin, et autres
Publié: (2025)

Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks
par: Zhang, Chenyang, et autres
Publié: (2025)

Transformers Provably Learn Sparse XOR with Polylogarithmic Parameters
par: Han, Yaomengxi, et autres
Publié: (2025)

The Implicit Bias of Adam on Separable Data
par: Zhang, Chenyang, et autres
Publié: (2024)

Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks
par: Nichani, Eshaan, et autres
Publié: (2023)

A Single-Loop Deep Actor-Critic Algorithm for Constrained Reinforcement Learning with Provable Convergence
par: Wang, Kexuan, et autres
Publié: (2023)

Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Role of Bellman Constraints
par: Xu, Tian, et autres
Publié: (2026)

An Efficient Subgraph GNN with Provable Substructure Counting Power
par: Yan, Zuoyu, et autres
Publié: (2023)

Unlabeled Data Can Provably Enhance In-Context Learning of Transformers
par: Liu, Renpu, et autres
Publié: (2026)

Energy-Entropy Regularization: The True Power of Minimal Looped Transformers
par: Lam, Wai-Lun
Publié: (2026)

Understanding the Role of Layer Normalization in Label-Skewed Federated Learning
par: Zhang, Guojun, et autres
Publié: (2023)

Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis
par: Li, Hongkang, et autres
Publié: (2024)

Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models
par: Wu, Yuchen, et autres
Publié: (2024)

Closing the Loop: A Control-Theoretic Framework for Provably Stable Time Series Forecasting with LLMs
par: Zhang, Xingyu, et autres
Publié: (2026)

Provably Transformers Harness Multi-Concept Word Semantics for Efficient In-Context Learning
par: Bu, Dake, et autres
Publié: (2024)

Parallel Layer Normalization for Universal Approximation
par: Ni, Yunhao, et autres
Publié: (2025)

Momentum Benefits Non-IID Federated Learning Simply and Provably
par: Cheng, Ziheng, et autres
Publié: (2023)

On the Robustness of Transformers against Context Hijacking for Linear Classification
par: Li, Tianle, et autres
Publié: (2025)

Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information
par: Cheng, Yuan, et autres
Publié: (2025)

Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks
par: Collins, Liam, et autres
Publié: (2023)

Towards Understanding Transformers in Learning Random Walks
par: Shi, Wei, et autres
Publié: (2025)

On Provable Benefits of Muon in Federated Learning
par: Zhang, Xinwen, et autres
Publié: (2025)

The Hidden Power of Normalization Layers in Neural Networks: Exponential Capacity Control
par: Than, Khoat
Publié: (2025)

Reasoning with Latent Thoughts: On the Power of Looped Transformers
par: Saunshi, Nikunj, et autres
Publié: (2025)

Achieving More with Less: A Tensor-Optimization-Powered Ensemble Method
par: Yuan, Jinghui, et autres
Publié: (2024)

Transformers Provably Learn Algorithmic Solutions for Graph Connectivity, But Only with the Right Data
par: Ye, Qilin, et autres
Publié: (2025)

On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding
par: Xu, Kevin, et autres
Publié: (2024)