Image de couverture de livre

Token Sample Complexity of Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bohbot, Léa, Letrouit, Cyril, Peyré, Gabriel, Vialard, François-Xavier
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime
par: Barboni, Raphaël, et autres
Publié: (2025)

Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport
par: Barboni, Raphaël, et autres
Publié: (2024)

Sliced ReLU attention: Quasi-linear contextual expressivity via sorting
par: Vialard, François-Xavier, et autres
Publié: (2025)

Towards Understanding the Universality of Transformers for Next-Token Prediction
par: Sander, Michael E., et autres
Publié: (2024)

How Smooth Is Attention?
par: Castin, Valérie, et autres
Publié: (2023)

Understanding the training of infinitely deep and wide ResNets with conditional optimal transport
par: Raphaël Barboni, et autres
Publié: (2025)

Robust Sublinear Convergence Rates for Iterative Bregman Projections
par: Peyré, Gabriel
Publié: (2026)

The emergence of clusters in self-attention dynamics
par: Geshkovski, Borjan, et autres
Publié: (2023)

Optimal and Diffusion Transports in Machine Learning
par: Peyré, Gabriel
Publié: (2025)

Optimal Transport for Machine Learners
par: Peyré, Gabriel
Publié: (2025)

Muon Dynamics as a Spectral Wasserstein Flow
par: Peyré, Gabriel
Publié: (2026)

A mathematical perspective on Transformers
par: Geshkovski, Borjan, et autres
Publié: (2023)

Learning from Samples: Inverse Problems over measures via Sharpened Fenchel-Young Losses
par: Andrade, Francisco, et autres
Publié: (2025)

Decreasing Entropic Regularization Averaged Gradient for Semi-Discrete Optimal Transport
par: Genans, Ferdinand, et autres
Publié: (2025)

Semi-Discrete Optimal Transport: Nearly Minimax Estimation With Stochastic Gradient Descent and Adaptive Entropic Regularization
par: Genans, Ferdinand, et autres
Publié: (2024)

Stochastic Optimization in Semi-Discrete Optimal Transport: Convergence Analysis and Minimax Rate
par: Genans, Ferdinand, et autres
Publié: (2025)

Intrinsic training dynamics of deep neural networks
par: Marcotte, Sibylle, et autres
Publié: (2025)

Transformative or Conservative? Conservation laws for ResNets and Transformers
par: Marcotte, Sibylle, et autres
Publié: (2025)

Geometry-Aware Discretization Error of Diffusion Models
par: Hurault, Samuel, et autres
Publié: (2026)

Unstable optimal transport maps
par: Letrouit, Cyril
Publié: (2025)

Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows
par: Marcotte, Sibylle, et autres
Publié: (2023)

On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
par: Petit, Romain, et autres
Publié: (2026)

Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
par: Marcotte, Sibylle, et autres
Publié: (2024)

Enhancing Hypergradients Estimation: A Study of Preconditioning and Reparameterization
par: Ye, Zhenzhang, et autres
Publié: (2024)

Transformers are Universal In-context Learners
par: Furuya, Takashi, et autres
Publié: (2024)

Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
par: Castin, Valérie, et autres
Publié: (2026)

How do Transformers perform In-Context Autoregressive Learning?
par: Sander, Michael E., et autres
Publié: (2024)

From Score Matching to Diffusion: A Fine-Grained Error Analysis in the Gaussian Setting
par: Hurault, Samuel, et autres
Publié: (2025)

Gluing methods for quantitative stability of optimal transport maps
par: Letrouit, Cyril, et autres
Publié: (2024)

Delocalized eigenvectors of transitive graphs and beyond
par: Burq, Nicolas, et autres
Publié: (2024)

Maximal multiplicity of Laplacian eigenvalues in negatively curved surfaces
par: Letrouit, Cyril, et autres
Publié: (2023)

Generic controllability of equivariant systems and applications to particle systems and neural networks
par: Agrachev, Andrei, et autres
Publié: (2024)

Benign Overfitting in Token Selection of Attention Mechanism
par: Sakamoto, Keitaro, et autres
Publié: (2024)

Training Infinitely Deep and Wide Transformers
par: Barboni, Raphaël, et autres
Publié: (2026)

Token Distillation: Attention-aware Input Embeddings For New Tokens
par: Dobler, Konstantin, et autres
Publié: (2025)

NI Sampling: Accelerating Discrete Diffusion Sampling by Token Order Optimization
par: Liu, Enshu, et autres
Publié: (2026)

Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective
par: Yan, Fanqi, et autres
Publié: (2025)

Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding
par: Pham, Duy-Tung, et autres
Publié: (2025)

Interdomain Attention: Beyond Token-Level Key-Value Memory
par: Kiyohara, Naoki, et autres
Publié: (2026)

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
par: Jo, Dongwon, et autres
Publié: (2026)