Imagen de Portada

Training Infinitely Deep and Wide Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Barboni, Raphaël, de Hoop, Maarten V., Furuya, Takashi, Peyré, Gabriel
Formato:	Preprint
Publicado:	2026
Materias:	Optimization and Control Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport
por: Barboni, Raphaël, et al.
Publicado: (2024)

Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime
por: Barboni, Raphaël, et al.
Publicado: (2025)

Muon Dynamics as a Spectral Wasserstein Flow
por: Peyré, Gabriel
Publicado: (2026)

Optimal and Diffusion Transports in Machine Learning
por: Peyré, Gabriel
Publicado: (2025)

Optimal Transport for Machine Learners
por: Peyré, Gabriel
Publicado: (2025)

Transformers are Universal In-context Learners
por: Furuya, Takashi, et al.
Publicado: (2024)

The Mathematics of Artificial Intelligence
por: Peyré, Gabriel
Publicado: (2025)

Robust Sublinear Convergence Rates for Iterative Bregman Projections
por: Peyré, Gabriel
Publicado: (2026)

Infinite-Horizon Reach-Avoid Zero-Sum Games via Deep Reinforcement Learning
por: Li, Jingqi, et al.
Publicado: (2022)

The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training
por: Wang, Jinbo, et al.
Publicado: (2025)

A Convexity-dependent Two-Phase Training Algorithm for Deep Neural Networks
por: Hrycej, Tomas, et al.
Publicado: (2025)

Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization
por: Chen, Zixiang, et al.
Publicado: (2025)

On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
por: Petit, Romain, et al.
Publicado: (2026)

Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows
por: Marcotte, Sibylle, et al.
Publicado: (2023)

Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
por: Marcotte, Sibylle, et al.
Publicado: (2024)

Anytime Training with Schedule-Free Spectral Optimization
por: Apte, Anuj, et al.
Publicado: (2026)

Accelerating RLHF Training with Reward Variance Increase
por: Yang, Zonglin, et al.
Publicado: (2025)

Stability of Transformers under Layer Normalization
por: Kan, Kelvin, et al.
Publicado: (2025)

Spatial Transformers for Radio Map Estimation
por: Viet, Pham Q., et al.
Publicado: (2024)

Training Safe Neural Networks with Global SDP Bounds
por: Soletskyi, Roman, et al.
Publicado: (2024)

Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
por: Sheng, Jiayuan, et al.
Publicado: (2025)

A multilevel approach to accelerate the training of Transformers
por: Lauga, Guillaume, et al.
Publicado: (2025)

Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization
por: Zhang, Shihao, et al.
Publicado: (2025)

YuriiFormer: A Suite of Nesterov-Accelerated Transformers
por: Zimin, Aleksandr, et al.
Publicado: (2026)

Finite-Time Analysis of Gradient Descent for Shallow Transformers
por: Arda, Enes, et al.
Publicado: (2026)

Understanding Optimization in Deep Learning with Central Flows
por: Cohen, Jeremy M., et al.
Publicado: (2024)

Deep Reinforcement Learning for Traveling Purchaser Problems
por: Yuan, Haofeng, et al.
Publicado: (2024)

Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
por: Chen, Siyu, et al.
Publicado: (2024)

The Optimiser Hidden in Plain Sight: Training with the Loss Landscape's Induced Metric
por: Harvey, Thomas R.
Publicado: (2025)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
por: Meterez, Alexandru, et al.
Publicado: (2025)

Federated Dynamical Low-Rank Training with Global Loss Convergence Guarantees
por: Schotthöfer, Steffen, et al.
Publicado: (2024)

Agentic Transformers Provably Learn to Search via Reinforcement Learning
por: Yang, Tong, et al.
Publicado: (2026)

Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
por: Kan, Kelvin, et al.
Publicado: (2025)

How Well Can Transformers Emulate In-context Newton's Method?
por: Giannou, Angeliki, et al.
Publicado: (2024)

Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization
por: Huang, Yu, et al.
Publicado: (2025)

Deep Learning for Two-Stage Robust Integer Optimization
por: Dumouchelle, Justin, et al.
Publicado: (2023)

Solving Functional Optimization with Deep Networks and Variational Principles
por: Kamtue, Kawisorn, et al.
Publicado: (2024)

Unsupervised Training of Diffusion Models for Feasible Solution Generation in Neural Combinatorial Optimization
por: Hong, Seong-Hyun, et al.
Publicado: (2024)

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
por: Song, Minhak, et al.
Publicado: (2025)

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
por: Li, Zihao, et al.
Publicado: (2024)