Image de couverture de livre

Quantitative Clustering in Mean-Field Transformer Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chen, Shi, Lin, Zhengjiang, Polyanskiy, Yury, Rigollet, Philippe
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Analysis of PDEs Dynamical Systems
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

A mathematical perspective on Transformers
par: Geshkovski, Borjan, et autres
Publié: (2023)

Synchronization of mean-field models on the circle
par: Polyanskiy, Yury, et autres
Publié: (2025)

Dynamic metastability in the self-attention model
par: Geshkovski, Borjan, et autres
Publié: (2024)

Clustering in Causal Attention Masking
par: Karagodin, Nikita, et autres
Publié: (2024)

Residual connections provably mitigate oversmoothing in graph neural networks
par: Chen, Ziang, et autres
Publié: (2025)

The Mean-Field Dynamics of Transformers
par: Rigollet, Philippe
Publié: (2025)

The emergence of clusters in self-attention dynamics
par: Geshkovski, Borjan, et autres
Publié: (2023)

Critical attention scaling in long-context transformers
par: Chen, Shi, et autres
Publié: (2025)

Propagation of Chaos in Contextual Flow Maps
par: Chen, Shi, et autres
Publié: (2026)

Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
par: Alcalde, Albert, et autres
Publié: (2026)

Scaling Limits of Long-Context Transformers
par: Bruno, Giuseppe, et autres
Publié: (2026)

Weak-Strong Uniqueness for Second-Order Mean-Field Games
par: Ferreira, Rita, et autres
Publié: (2026)

Existence of Solutions and Selection Problem for Quasi-stationary Contact Mean Field Games
par: Hu, Xiaotian
Publié: (2025)

Latent Space Element Method
par: Chung, Seung Whan, et autres
Publié: (2026)

Adaptive control of reaction-diffusion PDEs via neural operator-approximated gain kernels
par: Bhan, Luke, et autres
Publié: (2024)

Mean Field Games without Rational Expectations
par: Moll, Benjamin, et autres
Publié: (2025)

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
par: Isobe, Noboru, et autres
Publié: (2026)

Information theory and discriminative sampling for model discovery
par: Bao, Yuxuan, et autres
Publié: (2025)

Smooth Piecewise Cutting for Neural Operator to Handle Discontinuities and Sharp Transitions
par: Dang, Ha, et autres
Publié: (2026)

Mean field limit for interacting systems on co-evolving networks
par: Throm, Sebastian
Publié: (2025)

The Mean-Field Ott-Antonsen Manifold is an Unstable Manifold in the Continuum Limit
par: Kuehn, Christian, et autres
Publié: (2025)

Universality of almost periodic orbits in certain composite functions
par: Nakayama, Chikara, et autres
Publié: (2023)

Error Estimation in the Mean-Field Limit of Kinetic Flocking Models with Local Alignments
par: Wang, Jinhuan, et autres
Publié: (2024)

Quantitative estimates for flows of regular Lagrangian flows for Hörmander singular kernels and LD vector fields
par: Borrin, Henrique
Publié: (2025)

Computer-Assisted Proofs of Gap Solitons in Bose-Einstein Condensates
par: Ayala, Miguel, et autres
Publié: (2025)

Stationary non-radial localized patterns in the planar Swift-Hohenberg PDE: constructive proofs of existence
par: Cadiot, Matthieu, et autres
Publié: (2024)

Rigorous computation of solutions of semi-linear PDEs on unbounded domains via spectral methods
par: Cadiot, Matthieu, et autres
Publié: (2023)

Periodic localized traveling waves in the two-dimensional suspension bridge equation
par: van der Aalst, Lindsey, et autres
Publié: (2024)

Quantitative phase mixing for Hamiltonians with trapping
par: Hadžić, Mahir, et autres
Publié: (2024)

Optimal transport in the frame of abstract Lax-Oleinik operator revisited
par: Cheng, Wei, et autres
Publié: (2024)

KAM for high-dimensional nonlinear quantum harmonic oscillator
par: Liu, Jianjun, et autres
Publié: (2024)

Singularities and their propagation in optimal transport
par: Cannarsa, Piermarco, et autres
Publié: (2025)

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning
par: Faroughi, Salah A, et autres
Publié: (2026)

YuriiFormer: A Suite of Nesterov-Accelerated Transformers
par: Zimin, Aleksandr, et autres
Publié: (2026)

A new selection problem for degenerate viscous Hamilton-Jacobi equations
par: Chen, Qinbo, et autres
Publié: (2026)

Linearly Stable KAM Tori for One Dimensional Forced Kirchhoff Equations under Periodic Boundary Conditions
par: Chen, Yin, et autres
Publié: (2025)

Localized Pattern Formation and Oscillatory Instabilities in a Three-component Gierer Meinhardt Model
par: Gai, Chunyi, et autres
Publié: (2025)

Plankton-Oxygen Dynamics in the Context of Climate Change: A Fractional Model with A Probability Density Function Approach
par: El-Borai, Mahmoud M., et autres
Publié: (2025)

Existence and Stability Theory of a Neurologically-Inspired Parabolic PDE Model with a Nonlinear Time-Delayed Boundary Condition
par: Boregowda, Gangadhara, et autres
Publié: (2025)

Mean-Square Exponential Stabilization of Mixed-Autonomy Traffic PDE System
par: Zhang, Yihuai, et autres
Publié: (2023)