Cover Image

Clustering in Causal Attention Masking

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Karagodin, Nikita, Polyanskiy, Yury, Rigollet, Philippe
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Analysis of PDEs Dynamical Systems 68T07, 35Q68, 37N99, 82C22
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Normalization in Attention Dynamics
by: Karagodin, Nikita, et al.
Published: (2025)

Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
by: Alcalde, Albert, et al.
Published: (2026)

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
by: Isobe, Noboru, et al.
Published: (2026)

ARDO: A Weak Formulation Deep Neural Network Method for Elliptic and Parabolic PDEs Based on Random Differences of Test Functions
by: Cai, Wei, et al.
Published: (2025)

Error estimates of asymptotic-preserving neural networks in approximating stochastic linearized Boltzmann equation
by: Wan, Jiayu, et al.
Published: (2025)

Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regime
by: Chizat, Lénaïc, et al.
Published: (2025)

Machine Learning-based quadratic closures for non-intrusive Reduced Order Models
by: Codega, Gabriele, et al.
Published: (2025)

Transformer-based Koopman Autoencoder for Linearizing Fisher's Equation
by: Rana, Kanav Singh, et al.
Published: (2024)

The Compute ICE-AGE: Invariant Compute Envelope under Addressable Graph Evolution
by: Martin II, R. Jay
Published: (2026)

Computational Complexity of Minimal Trap Spaces in Boolean Networks
by: Moon, Kyungduk, et al.
Published: (2022)

Data-driven Identification of Attractors Using Machine Learning
by: Gameiro, Marcio, et al.
Published: (2025)

A Physics-Informed, Global-in-Time Neural Particle Method for the Spatially Homogeneous Landau Equation
by: Kim, Minseok, et al.
Published: (2026)

Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEs
by: Wöhrer, Tobias, et al.
Published: (2026)

The lowest-order Neural Approximated Virtual Element Method on polygonal elements
by: Berrone, Stefano, et al.
Published: (2024)

A Formal Analysis of Iterated TDD
by: Ruparel, Hemil, et al.
Published: (2024)

Sampling Boltzmann distributions via normalizing flow approximation of transport maps
by: Rehman, Zia Ur, et al.
Published: (2026)

Ergodic Generative Flows
by: Brunswic, Leo Maxime, et al.
Published: (2025)

Solving Inverse PDE Problems using Minimization Methods and AI
by: Helwani, Noura Al, et al.
Published: (2026)

Memorisation and forgetting in a learning Hopfield neural network: bifurcation mechanisms, attractors and basins
by: Essex, Adam E., et al.
Published: (2025)

Quantitative Convergence of Wasserstein Gradient Flows of Kernel Mean Discrepancies
by: Chizat, Lénaïc, et al.
Published: (2026)

Next Generation Equation-Free Multiscale Modelling of Crowd Dynamics via Machine Learning
by: Alvarez, Hector Vargas, et al.
Published: (2025)

Diffeomorphic Measure Matching with Kernels for Generative Modeling
by: Pandey, Biraj, et al.
Published: (2024)

Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions
by: Hess-Childs, Elias, et al.
Published: (2026)

On the Dynamical Hierarchy in Gathering Protocols with Circulant Topologies
by: Gerlach, Raphael, et al.
Published: (2023)

SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds
by: Stein, Viktor, et al.
Published: (2026)

Wasserstein-Łojasiewicz inequalities and asymptotics of McKean-Vlasov equation
by: Choi, Beomjun, et al.
Published: (2025)

A notion of partial order in the Choose the Leader model
by: Einav, Amit, et al.
Published: (2025)

Stability and bifurcation for the Kuramoto model
by: Dietert, Helge
Published: (2014)

A PDE Perspective on Generative Diffusion Models
by: Liu, Kang, et al.
Published: (2025)

Entropic convergence and the linearized limit for the Boltzmann equation with external force
by: Mai, Tina
Published: (2016)

Localmax dynamics for attention in transformers and its asymptotic behavior
by: Cimetière, Henri, et al.
Published: (2025)

Perturbation Analysis and Neural Network-Based Initial Condition Estimation for the Sine-Gordon Equation
by: Ha, Junhong, et al.
Published: (2025)

On the convergence of PINNs for inverse source problem in the complex Ginzburg-Landau equation
by: Cheng, Xing, et al.
Published: (2025)

Dynamic Level Sets
by: Fiske, Michael Stephen
Published: (2026)

A Unified Perspective on the Dynamics of Deep Transformers
by: Castin, Valérie, et al.
Published: (2025)

Existence of traveling wave solutions in continuous OV models
by: Ikeda, Kota, et al.
Published: (2024)

Solving nonconvex Hamilton--Jacobi--Isaacs equations with PINN-based policy iteration
by: Yang, Hee Jun, et al.
Published: (2025)

On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers
by: Balasubramanian, Krishnakumar, et al.
Published: (2025)

Cluster-based classification with neural ODEs via control
by: Álvarez-López, Antonio, et al.
Published: (2023)

Non-Robustness of the Zero-Temperature-Limit Gibbs Measures to Perturbations of the Potential
by: Gayral, Léo, et al.
Published: (2025)