Imagen de Portada

Towards Understanding the Universality of Transformers for Next-Token Prediction

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Sander, Michael E., Peyré, Gabriel
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

How do Transformers perform In-Context Autoregressive Learning?
por: Sander, Michael E., et al.
Publicado: (2024)

Transformers are Universal In-context Learners
por: Furuya, Takashi, et al.
Publicado: (2024)

Transformative or Conservative? Conservation laws for ResNets and Transformers
por: Marcotte, Sibylle, et al.
Publicado: (2025)

Token Sample Complexity of Attention
por: Bohbot, Léa, et al.
Publicado: (2025)

Robust Sublinear Convergence Rates for Iterative Bregman Projections
por: Peyré, Gabriel
Publicado: (2026)

Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Prediction
por: Blondel, Mathieu, et al.
Publicado: (2025)

Muon Dynamics as a Spectral Wasserstein Flow
por: Peyré, Gabriel
Publicado: (2026)

Optimal and Diffusion Transports in Machine Learning
por: Peyré, Gabriel
Publicado: (2025)

Optimal Transport for Machine Learners
por: Peyré, Gabriel
Publicado: (2025)

Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport
por: Barboni, Raphaël, et al.
Publicado: (2024)

Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
por: Rofin, Mark, et al.
Publicado: (2026)

How Smooth Is Attention?
por: Castin, Valérie, et al.
Publicado: (2023)

Intrinsic training dynamics of deep neural networks
por: Marcotte, Sibylle, et al.
Publicado: (2025)

Geometry-Aware Discretization Error of Diffusion Models
por: Hurault, Samuel, et al.
Publicado: (2026)

Cautious Next Token Prediction
por: Wang, Yizhou, et al.
Publicado: (2025)

Trajeglish: Traffic Modeling as Next-Token Prediction
por: Philion, Jonah, et al.
Publicado: (2023)

Generative Verifiers: Reward Modeling as Next-Token Prediction
por: Zhang, Lunjun, et al.
Publicado: (2024)

Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
por: Gong, Zixuan, et al.
Publicado: (2025)

ENTP: Encoder-only Next Token Prediction
por: Ewer, Ethan, et al.
Publicado: (2024)

Reasoning Bias of Next Token Prediction Training
por: Lin, Pengxiao, et al.
Publicado: (2025)

Training Infinitely Deep and Wide Transformers
por: Barboni, Raphaël, et al.
Publicado: (2026)

Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
por: Marcotte, Sibylle, et al.
Publicado: (2024)

Learning from Samples: Inverse Problems over measures via Sharpened Fenchel-Young Losses
por: Andrade, Francisco, et al.
Publicado: (2025)

Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows
por: Marcotte, Sibylle, et al.
Publicado: (2023)

On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
por: Petit, Romain, et al.
Publicado: (2026)

Auto-Regressive Next-Token Predictors are Universal Learners
por: Malach, Eran
Publicado: (2023)

Scaling Next-Brain-Token Prediction for MEG
por: Csaky, Richard
Publicado: (2026)

Humanoid Locomotion as Next Token Prediction
por: Radosavovic, Ilija, et al.
Publicado: (2024)

Provable Long-Range Benefits of Next-Token Prediction
por: Cao, Xinyuan, et al.
Publicado: (2025)

Enhancing Hypergradients Estimation: A Study of Preconditioning and Reparameterization
por: Ye, Zhenzhang, et al.
Publicado: (2024)

How Reinforcement Learning After Next-Token Prediction Facilitates Learning
por: Tsilivis, Nikolaos, et al.
Publicado: (2025)

SENTRA: Selected-Next-Token Transformer for LLM Text Detection
por: Plyler, Mitchell, et al.
Publicado: (2025)

Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime
por: Barboni, Raphaël, et al.
Publicado: (2025)

Next-Token Prediction and Regret Minimization
por: Mohri, Mehryar, et al.
Publicado: (2026)

Implicit Optimization Bias of Next-Token Prediction in Linear Models
por: Thrampoulidis, Christos
Publicado: (2024)

Adaptively Private Next-Token Prediction of Large Language Models
por: Flemings, James, et al.
Publicado: (2024)

Prot2Token: A Unified Framework for Protein Modeling via Next-Token Prediction
por: Pourmirzaei, Mahdi, et al.
Publicado: (2025)

Mechanics of Next Token Prediction with Self-Attention
por: Li, Yingcong, et al.
Publicado: (2024)

Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
por: Castin, Valérie, et al.
Publicado: (2026)

Improving Next Tokens via Second-to-Last Predictions with Generate and Refine
por: Schneider, Johannes
Publicado: (2024)