Cover Image

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Isobe, Noboru, Inoue, Daisuke, Imaizumi, Masaaki
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Analysis of PDEs Dynamical Systems Optimization and Control 49N80, 35Q84, 49Q22, 68T07
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Implicit Regularization of Large Neural Networks via Mean-Field Formulation
by: Acciaio, Beatrice, et al.
Published: (2026)

On the equivalence between static and dynamic optimal transport governed by linear control systems
by: Einav, Amit, et al.
Published: (2025)

Quantitative Convergence of Wasserstein Gradient Flows of Kernel Mean Discrepancies
by: Chizat, Lénaïc, et al.
Published: (2026)

Variational formulations of ODE-Net as a mean-field optimal control problem and existence results
by: Isobe, Noboru, et al.
Published: (2023)

A convergence result of a continuous model of deep learning via Łojasiewicz--Simon inequality
by: Isobe, Noboru
Published: (2023)

On Mean Field Games in Infinite Dimension
by: Federico, Salvatore, et al.
Published: (2024)

Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equation
by: Isobe, Noboru, et al.
Published: (2024)

Transport based particle methods for the Fokker-Planck-Landau equation
by: Ilin, Vasily, et al.
Published: (2024)

Convergence of drift-diffusion PDEs arising as Wasserstein gradient flows of convex functions
by: Chizat, Lénaïc, et al.
Published: (2025)

New Uniqueness Results For A Mean Field Game Of Controls
by: Graber, Jameson, et al.
Published: (2024)

Policy Optimization for Continuous-time Linear-Quadratic Graphon Mean Field Games
by: Plank, Philipp, et al.
Published: (2025)

A Model Problem for First Order Mean Field Games with Discrete Initial Data
by: Graber, P. Jameson, et al.
Published: (2024)

Cascade equation for the discontinuities in the Stefan problem with surface tension
by: Guo, Yucheng, et al.
Published: (2024)

Variational structures for the Fokker--Planck equation with general Dirichlet boundary conditions
by: Quattrocchi, Filippo
Published: (2024)

Sharp upper bound for a branched transport problem coming from Ginzburg-Landau models
by: Cosenza, Alessandro, et al.
Published: (2026)

Ahlfors-regularity for minimizers of a multiphase optimal design problem
by: Esposito, Luca, et al.
Published: (2025)

LieDetect: Detection of representation orbits of compact Lie groups from point clouds
by: Ennes, Henrique, et al.
Published: (2023)

Regularization of Stationary Second-order Mean Field Game Partial Differential Inclusions
by: Osborne, Yohance A. P., et al.
Published: (2024)

On the equivalence of static and dynamic weak optimal transport
by: Bulanyi, Bohdan
Published: (2023)

Learning Mean-Field Games through Mean-Field Actor-Critic Flow
by: Zhou, Mo, et al.
Published: (2025)

Variational models of robust optimal transport
by: De Masi, Luigi, et al.
Published: (2025)

Local Exponential Stability of Mean-Field Langevin Descent-Ascent in Wasserstein Space
by: Seo, Geuntaek, et al.
Published: (2026)

A network model for urban planning
by: Camilli, Fabio, et al.
Published: (2024)

Multi-Headed Transformer Architectures as Time-dependent Wasserstein Gradient Flows
by: Massucco, Alex, et al.
Published: (2026)

SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds
by: Stein, Viktor, et al.
Published: (2026)

On the Regularity of a Weak Formulation of Stochastic Differential Mean-Field Games
by: Morgado, Hector Sanchez, et al.
Published: (2023)

A Mollification Approach to Ramified Transport and Tree Shape Optimization
by: Bressan, Alberto, et al.
Published: (2026)

A Blob Method for Mean Field Control With Terminal Constraints
by: Craig, Katy, et al.
Published: (2024)

A PDE approach to Benamou--Brenier formula for the Schrödinger problem
by: Garatti, Mattia, et al.
Published: (2026)

No-Regret Generative Modeling via Parabolic Monge-Ampère PDE
by: Deb, Nabarun, et al.
Published: (2025)

Constrained Mean Field Games with Grushin type dynamics
by: Cutrì, Alessandra, et al.
Published: (2026)

Meshless Shape Optimization using Neural Networks and Partial Differential Equations on Graphs
by: Martinet, Eloi, et al.
Published: (2025)

Constrained Hellinger-Kantorovich barycenters: least-cost soft and conic multi-marginal formulations
by: Buze, Maciej
Published: (2024)

Fixed-Point Neural Optimal Transport without Implicit Differentiation
by: Park, Yesom, et al.
Published: (2026)

Deterministic Mean Field Games on Networks and Related Optimal Control Problems
by: Achdou, Yves, et al.
Published: (2025)

Discretization and Vanishing Discount Problems for First-order Mean Field Games
by: Iturriaga, Renato, et al.
Published: (2025)

Sparsity and uniform regularity for regularised optimal transport
by: Gvalani, Rishabh S., et al.
Published: (2026)

Hybrid Optimization Techniques for Multi-State Optimal Design Problems
by: Erceg, Marko, et al.
Published: (2026)

Fully nonlinear second-order mean field games with nondifferentiable Hamiltonians
by: Sales, Thomas, et al.
Published: (2025)

Lipschitz-regularized gradient flows and generative particle algorithms for high-dimensional scarce data
by: Gu, Hyemin, et al.
Published: (2022)