Cover Image

How transformers learn structured data: insights from hierarchical filtering

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Garnier-Brun, Jerome, Mézard, Marc, Moscato, Emanuele, Saglietti, Luca
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Biased Generalization in Diffusion Models
by: Garnier-Brun, Jerome, et al.
Published: (2026)

Replication and Information Extraction in a Minimal Agent-Environment Model
by: Ariosto, Sebastiano, et al.
Published: (2025)

Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
by: Bocchi, Dario, et al.
Published: (2026)

Unlearnable Games and "Satisficing'' Decisions: A Simple Model for a Complex World
by: Garnier-Brun, Jerome, et al.
Published: (2023)

Distinct mechanisms underlying in-context learning in transformers
by: Gibson, Cole, et al.
Published: (2026)

On the structure factor of jammed particle configurations on the one-dimensional lattice
by: Luck, Jean-Marc
Published: (2023)

Kernel Density Estimators in Large Dimensions
by: Biroli, Giulio, et al.
Published: (2024)

Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
by: Giorlandino, Alessio, et al.
Published: (2026)

A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
by: Ricci, Fabiola, et al.
Published: (2026)

Compressed sensing with l0-norm: statistical physics analysis and algorithms for signal recovery
by: Barbier, D., et al.
Published: (2023)

Investigating Stark many-body localization with continuous unitary transformation flows
by: Herre, Jan-Niklas, et al.
Published: (2024)

Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation
by: Giorlandino, Alessio, et al.
Published: (2025)

How does Chain of Thought decompose complex tasks?
by: Nadgir, Amrut, et al.
Published: (2026)

Coding schemes in neural networks learning classification tasks
by: van Meegen, Alexander, et al.
Published: (2024)

Localized and delocalized modes on random geometric graphs in 1D
by: Schaefer, Luca, et al.
Published: (2025)

Spring-block theory of feature learning in deep neural networks
by: Shi, Cheng, et al.
Published: (2024)

Preferential attachment and power-law degree distributions in heterogeneous multilayer hypergraphs
by: Di Lauro, Francesco, et al.
Published: (2025)

Phase determination with and without deep learning
by: Çivitcioğlu, Burak, et al.
Published: (2024)

Machine learning the Ising transition: A comparison between discriminative and generative approaches
by: Zhang, Difei, et al.
Published: (2024)

Hopfield model for patterns with internal structure
by: Nieuwenhuizen, Theodorus Maria
Published: (2026)

Estimating predictability of depinning dynamics by machine learning
by: Haavisto, Valtteri, et al.
Published: (2023)

Intensity pseudo-localized phase in the glassy random laser
by: Niedda, Jacopo, et al.
Published: (2022)

Microscopic and collective signatures of feature learning in neural networks
by: Corti, Andrea, et al.
Published: (2025)

Beyond dynamic scaling: rare events break universality
by: Marquis, Ulysse, et al.
Published: (2026)

Phase probabilities in first-order transitions using machine learning
by: Sukhoverkhova, Diana, et al.
Published: (2024)

What makes it possible to learn probability distributions in the natural world?
by: Bialek, William, et al.
Published: (2020)

Locality versus Fock-space structure in East-type models
by: Lazarides, Achilleas
Published: (2026)

Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
by: D'Amico, Francesco, et al.
Published: (2025)

Statistical mechanics of transfer learning in fully-connected networks in the proportional limit
by: Ingrosso, Alessandro, et al.
Published: (2024)

Analytical solution to Heisenberg spin glass models on sparse random graphs and their de Almeida-Thouless line
by: Del Bono, Luca Maria, et al.
Published: (2024)

Mapping of attention mechanisms to a generalized Potts model
by: Rende, Riccardo, et al.
Published: (2023)

The twin peaks of learning neural networks
by: Demyanenko, Elizaveta, et al.
Published: (2024)

Machine learning analysis of dimensional reduction conjecture for nonequilibrium Berezinskii-Kosterlitz-Thouless transition in three dimensions
by: Haga, Taiki
Published: (2023)

Machine learning stochastic differential equations for the evolution of order parameters of classical many-body systems in and out of equilibrium
by: Carnazza, Francesco, et al.
Published: (2024)

Impact of ChatGPT on the writing style of condensed matter physicists
by: Xu, Shaojun, et al.
Published: (2024)

Diagrammatics of free energies with fixed variance for high-dimensional data
by: Kühn, Tobias
Published: (2025)

Dataset-learning duality and emergent criticality
by: Kukleva, Ekaterina, et al.
Published: (2024)

Tree networks of real-world data: analysis of efficiency and spatiotemporal scales
by: Cipollini, Davide, et al.
Published: (2024)

Smoothed-Cubic Spin-Glass Model of Random Lasers
by: Benedetti, Marcello, et al.
Published: (2025)

Spin glass model of in-context learning
by: Li, Yuhao, et al.
Published: (2024)