Image de couverture de livre

When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Mousavi-Hosseini, Alireza, Sanford, Clayton, Wu, Denny, Erdogdu, Murat A.
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics
par: Mousavi-Hosseini, Alireza, et autres
Publié: (2024)

Post-Training with Policy Gradients: Optimality and the Base Model Barrier
par: Mousavi-Hosseini, Alireza, et autres
Publié: (2026)

Robust Feature Learning for Multi-Index Models in High Dimensions
par: Mousavi-Hosseini, Alireza, et autres
Publié: (2024)

From Information to Generative Exponent: Learning Rate Induces Phase Transitions in SGD
par: Tsiolis, Konstantinos Christopher, et autres
Publié: (2025)

A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers
par: He, Ye, et autres
Publié: (2024)

Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
par: Arous, Gérard Ben, et autres
Publié: (2025)

Transformers, parallel computation, and logarithmic depth
par: Sanford, Clayton, et autres
Publié: (2024)

Pruning is Optimal for Learning Sparse Features in High-Dimensions
par: Vural, Nuri Mert, et autres
Publié: (2024)

Optimal Excess Risk Bounds for Empirical Risk Minimization on $p$-Norm Linear Regression
par: Hanchi, Ayoub El, et autres
Publié: (2023)

A Geometric Analysis of PCA
par: Hanchi, Ayoub El, et autres
Publié: (2025)

When Do Multi-Agent Systems Outperform? Analysing the Learning Efficiency of Agentic Systems
par: Su, Junwei, et autres
Publié: (2026)

Mean-Field Langevin Dynamics for Signed Measures via a Bilevel Approach
par: Wang, Guillaume, et autres
Publié: (2024)

One-layer transformers fail to solve the induction heads task
par: Sanford, Clayton, et autres
Publié: (2024)

When Do Neural Nets Outperform Boosted Trees on Tabular Data?
par: McElfresh, Duncan, et autres
Publié: (2023)

On Fitting Flow Models with Large Sinkhorn Couplings
par: Zhang, Stephen, et autres
Publié: (2025)

Beyond Labeling Oracles: What does it mean to steal ML models?
par: Shafran, Avital, et autres
Publié: (2023)

Minimax Linear Regression under the Quantile Risk
par: Hanchi, Ayoub El, et autres
Publié: (2024)

On the Efficiency of ERM in Feature Learning
par: Hanchi, Ayoub El, et autres
Publié: (2024)

Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learning
par: Kovačević, Filip, et autres
Publié: (2026)

Flow Matching with Semidiscrete Couplings
par: Mousavi-Hosseini, Alireza, et autres
Publié: (2025)

When Independent Sampling Outperforms Agentic Reasoning
par: Dong, Yihe, et autres
Publié: (2026)

Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers
par: Bechler-Speicher, Maya, et autres
Publié: (2026)

A Statistical-Modelling Approach to Feedforward Neural Network Model Selection
par: McInerney, Andrew, et autres
Publié: (2022)

When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks
par: Wu, Haihang, et autres
Publié: (2024)

Attention Is Not All You Need: The Importance of Feedforward Networks in Transformer Models
par: Gerber, Isaac
Publié: (2025)

Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?
par: Chakraborty, Sanjay, et autres
Publié: (2025)

Sampling from the Mean-Field Stationary Distribution
par: Kook, Yunbum, et autres
Publié: (2024)

Analysis of Langevin Monte Carlo from Poincaré to Log-Sobolev
par: Chewi, Sinho, et autres
Publié: (2021)

Fast attention mechanisms: a tale of parallelism
par: Liu, Jingwen, et autres
Publié: (2025)

When Does Visual Prompting Outperform Linear Probing for Vision-Language Models? A Likelihood Perspective
par: Tsao, Hsi-Ai, et autres
Publié: (2024)

When Does Predictive Inverse Dynamics Outperform Behavior Cloning?
par: Schäfer, Lukas, et autres
Publié: (2026)

Approximation Capabilities of Feedforward Neural Networks with GELU Activations
par: Yakovlev, Konstantin, et autres
Publié: (2025)

Training Feedforward Neural Networks with Bayesian Hyper-Heuristics
par: Schreuder, Arné, et autres
Publié: (2023)

A Riemannian Optimization Perspective of the Gauss-Newton Method for Feedforward Neural Networks
par: Cayci, Semih
Publié: (2024)

Understanding Transformer Reasoning Capabilities via Graph Algorithms
par: Sanford, Clayton, et autres
Publié: (2024)

Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
par: Yehudai, Gilad, et autres
Publié: (2025)

When Less Is More: Binary Feedback Can Outperform Ordinal Comparisons in Ranking Recovery
par: Xu, Shirong, et autres
Publié: (2025)

Inheritance Between Feedforward and Convolutional Networks via Model Projection
par: Ewen, Nicolas, et autres
Publié: (2026)

Propagation of Chaos in One-hidden-layer Neural Networks beyond Logarithmic Time
par: Glasgow, Margalit, et autres
Publié: (2025)

Next-Token Prediction and Regret Minimization
par: Mohri, Mehryar, et autres
Publié: (2026)