Imagen de Portada

Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Dandi, Yatin, Vilucchio, Matteo, Arnaboldi, Luca, Tabanelli, Hugo, Krzakala, Florent
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Disordered Systems and Neural Networks
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Optimal Spectral Transitions in High-Dimensional Multi-Index Models
por: Defilippis, Leonardo, et al.
Publicado: (2025)

Fundamental limits of learning in sequence multi-index models and deep attention networks: High-dimensional asymptotics and sharp thresholds
por: Troiani, Emanuele, et al.
Publicado: (2025)

Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model
por: Tabanelli, Hugo, et al.
Publicado: (2025)

Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective
por: Ghio, Davide, et al.
Publicado: (2023)

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
por: Tanner, Kasimir, et al.
Publicado: (2024)

Asymptotics of feature learning in two-layer networks after one gradient-step
por: Cui, Hugo, et al.
Publicado: (2024)

Fundamental computational limits of weak learnability in high-dimensional multi-index models
por: Troiani, Emanuele, et al.
Publicado: (2024)

Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks
por: Arnaboldi, Luca, et al.
Publicado: (2025)

Deep Learning of Compositional Targets with Hierarchical Spectral Methods
por: Tabanelli, Hugo, et al.
Publicado: (2026)

Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions
por: Xu, Yizhou, et al.
Publicado: (2025)

Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method
por: Dandi, Yatin, et al.
Publicado: (2026)

Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
por: Defilippis, Leonardo, et al.
Publicado: (2025)

Spectral Phase Transition and Optimal PCA in Block-Structured Spiked models
por: Mergny, Pierre, et al.
Publicado: (2024)

Reentrant localisation transitions and anomalous spectral properties in off-diagonal quasiperiodic systems
por: Tabanelli, Hugo, et al.
Publicado: (2024)

On the existence of consistent adversarial attacks in high-dimensional linear classification
por: Vilucchio, Matteo, et al.
Publicado: (2025)

Quenches in the Sherrington-Kirkpatrick model
por: Erba, Vittorio, et al.
Publicado: (2024)

On the Geometry of Regularization in Adversarial Training: High-Dimensional Asymptotics and Generalization Bounds
por: Vilucchio, Matteo, et al.
Publicado: (2024)

Sequential Dynamics in Ising Spin Glasses
por: Dandi, Yatin, et al.
Publicado: (2025)

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression
por: Clarté, Lucas, et al.
Publicado: (2024)

Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
por: Boncoraglio, Fabrizio, et al.
Publicado: (2025)

The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks
por: Erba, Vittorio, et al.
Publicado: (2025)

Low-rank Matrix Estimation with Inhomogeneous Noise
por: Guionnet, Alice, et al.
Publicado: (2022)

Statistical Physics of Deep Neural Networks: Generalization Capability, Beyond the Infinite Width, and Feature Learning
por: Ariosto, Sebastiano
Publicado: (2025)

The phase diagram of compressed sensing with $\ell_0$-norm regularization
por: Barbier, Damien, et al.
Publicado: (2024)

From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning
por: Rubin, Noa, et al.
Publicado: (2025)

On the Atypical Solutions of the Symmetric Binary Perceptron
por: Barbier, Damien, et al.
Publicado: (2023)

Asymptotics of Learning with Deep Structured (Random) Features
por: Schröder, Dominik, et al.
Publicado: (2024)

Fundamental Limits of Matrix Sensing: Exact Asymptotics, Universality, and Applications
por: Xu, Yizhou, et al.
Publicado: (2025)

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
por: Lauditi, Clarissa, et al.
Publicado: (2026)

How Feature Learning Can Improve Neural Scaling Laws
por: Bordelon, Blake, et al.
Publicado: (2024)

Statistical mechanics of the maximum-average submatrix problem
por: Erba, Vittorio, et al.
Publicado: (2023)

How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model
por: Tomasini, Umberto, et al.
Publicado: (2024)

Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Model
por: Bordelon, Blake, et al.
Publicado: (2026)

Gaussian Universality of Perceptrons with Random Labels
por: Gerace, Federica, et al.
Publicado: (2022)

Bayes-optimal learning of an extensive-width neural network from quadratically many samples
por: Maillard, Antoine, et al.
Publicado: (2024)

Transfer Learning in Infinite Width Feature Learning Networks
por: Lauditi, Clarissa, et al.
Publicado: (2025)

Applications of Statistical Field Theory in Deep Learning
por: Ringel, Zohar, et al.
Publicado: (2025)

Learning Linear Regression with Low-Rank Tasks in-Context
por: Takanami, Kaito, et al.
Publicado: (2025)

Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures
por: Cagnetta, Francesco, et al.
Publicado: (2025)

Parameter Symmetry Potentially Unifies Deep Learning Theory
por: Ziyin, Liu, et al.
Publicado: (2025)