Cover Image

Dynamic metastability in the self-attention model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Geshkovski, Borjan, Koubbi, Hugo, Polyanskiy, Yury, Rigollet, Philippe
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Analysis of PDEs Dynamical Systems
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A mathematical perspective on Transformers
by: Geshkovski, Borjan, et al.
Published: (2023)

The emergence of clusters in self-attention dynamics
by: Geshkovski, Borjan, et al.
Published: (2023)

Synchronization of mean-field models on the circle
by: Polyanskiy, Yury, et al.
Published: (2025)

Quantitative Clustering in Mean-Field Transformer Models
by: Chen, Shi, et al.
Published: (2025)

Homogenized Transformers
by: Koubbi, Hugo, et al.
Published: (2026)

Clustering in Causal Attention Masking
by: Karagodin, Nikita, et al.
Published: (2024)

Kinetic theory for Transformers and the lost-in-the-middle phenomenon
by: Duerinckx, Mitia, et al.
Published: (2026)

Constructive conditional normalizing flows
by: Geshkovski, Borjan, et al.
Published: (2026)

Residual connections provably mitigate oversmoothing in graph neural networks
by: Chen, Ziang, et al.
Published: (2025)

Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics
by: Koubbi, Hugo, et al.
Published: (2024)

On the number of modes of Gaussian kernel density estimators
by: Geshkovski, Borjan, et al.
Published: (2024)

The Mean-Field Dynamics of Transformers
by: Rigollet, Philippe
Published: (2025)

Critical attention scaling in long-context transformers
by: Chen, Shi, et al.
Published: (2025)

Information theory and discriminative sampling for model discovery
by: Bao, Yuxuan, et al.
Published: (2025)

Latent Space Element Method
by: Chung, Seung Whan, et al.
Published: (2026)

Measure-to-measure interpolation using Transformers
by: Geshkovski, Borjan, et al.
Published: (2024)

Smooth Piecewise Cutting for Neural Operator to Handle Discontinuities and Sharp Transitions
by: Dang, Ha, et al.
Published: (2026)

Global solution and blow-up for the SKT model in Population Dynamics
by: Belkhamsa, Ichraf, et al.
Published: (2023)

Universality of almost periodic orbits in certain composite functions
by: Nakayama, Chikara, et al.
Published: (2023)

Adaptive control of reaction-diffusion PDEs via neural operator-approximated gain kernels
by: Bhan, Luke, et al.
Published: (2024)

Global self-similar solutions for Hardy-Hénon equations with linear and quasilinear diffusion
by: Iagar, Razvan Gabriel, et al.
Published: (2026)

Existence of blow-up self-similar solutions for the supercritical quasilinear reaction-diffusion equation
by: Iagar, Razvan Gabriel, et al.
Published: (2024)

Stationary non-radial localized patterns in the planar Swift-Hohenberg PDE: constructive proofs of existence
by: Cadiot, Matthieu, et al.
Published: (2024)

Computer-Assisted Proofs of Gap Solitons in Bose-Einstein Condensates
by: Ayala, Miguel, et al.
Published: (2025)

Rigorous computation of solutions of semi-linear PDEs on unbounded domains via spectral methods
by: Cadiot, Matthieu, et al.
Published: (2023)

Periodic localized traveling waves in the two-dimensional suspension bridge equation
by: van der Aalst, Lindsey, et al.
Published: (2024)

Shadowing for Infinite Dimensional Dynamical Systems
by: Arrieta, José M., et al.
Published: (2025)

Dynamics of globally minimizing orbits in contact Hamiltonian systems
by: Xu, Yang, et al.
Published: (2024)

Local Invariant Structures in the Dynamics of Capillary Water Jet
by: Shao, Chengyang, et al.
Published: (2026)

Learning Regions of Attraction in Unknown Dynamical Systems via Zubov-Koopman Lifting: Regularities and Convergence
by: Meng, Yiming, et al.
Published: (2023)

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning
by: Faroughi, Salah A, et al.
Published: (2026)

On the local stability of the elapsed-time model in terms of the transmission delay and interconnection strength
by: Cáceres, María J., et al.
Published: (2025)

Plankton-Oxygen Dynamics in the Context of Climate Change: A Fractional Model with A Probability Density Function Approach
by: El-Borai, Mahmoud M., et al.
Published: (2025)

Analysis of a nonlinear fish-bone model for suspension bridges with rigid hangers in the presence of flow effects
by: Falocchi, Alessio, et al.
Published: (2024)

An abstract framework for a class of nonlocal structured population models: existence, uniqueness and stability of steady states
by: Coville, Jérôme, et al.
Published: (2025)

Classification and stability analysis of polarising and depolarising travelling wave solutions for a model of collective cell migration
by: Rahman, Nizhum, et al.
Published: (2021)

Front propagation in hybrid reaction-diffusion epidemic models with spatial heterogeneity. Part I: Spreading speed and asymptotic behavior
by: Griette, Quentin, et al.
Published: (2024)

Finite-time blow-up prevention by logistic source in parabolic-elliptic chemotaxis models with singular sensitivity in any dimensional setting
by: Kurt, Halil Ibrahim, et al.
Published: (2020)

Infinite families of harmonic self-maps of ellipsoids in all dimensions
by: Branding, Volker, et al.
Published: (2022)

Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
by: Alcalde, Albert, et al.
Published: (2026)