Imagen de Portada

Implicit bias produces neural scaling laws in learning curves, from perceptrons to deep networks

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	D'Amico, Francesco, Bocchi, Dario, Negri, Matteo
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Disordered Systems and Neural Networks
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Self-attention as an attractor network: transient memories without backpropagation
por: D'Amico, Francesco, et al.
Publicado: (2024)

Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
por: D'Amico, Francesco, et al.
Publicado: (2025)

Statistical mechanics of vector Hopfield network near and above saturation
por: Nicoletti, Flavio, et al.
Publicado: (2025)

Escape dynamics and implicit bias of one-pass SGD in overparameterized quadratic networks
por: Bocchi, Dario, et al.
Publicado: (2026)

Benchmarking Graph Neural Networks in Solving Hard Constraint Satisfaction Problems
por: Skenderi, Geri, et al.
Publicado: (2026)

Renormalization group for deep neural networks: Universality of learning and scaling laws
por: Coppola, Gorka Peraza, et al.
Publicado: (2025)

Emergent weight morphologies in deep neural networks
por: de Jong, Pascal, et al.
Publicado: (2025)

Computing frustration and near-monotonicity in deep neural networks
por: Wendin, Joel, et al.
Publicado: (2025)

Finite-time Lyapunov exponents of deep neural networks
por: Storm, L., et al.
Publicado: (2023)

Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation
por: Barbier, Jean, et al.
Publicado: (2025)

Learning curves theory for hierarchically compositional data with power-law distributed features
por: Cagnetta, Francesco, et al.
Publicado: (2025)

High-dimensional learning of narrow neural networks
por: Cui, Hugo
Publicado: (2024)

Spring-block theory of feature learning in deep neural networks
por: Shi, Cheng, et al.
Publicado: (2024)

Towards a theory of how the structure of language is acquired by deep neural networks
por: Cagnetta, Francesco, et al.
Publicado: (2024)

Generative diffusion for perceptron problems: statistical physics analysis and efficient algorithms
por: Demyanenko, Elizaveta, et al.
Publicado: (2025)

Adaptive kernel predictors from feature-learning infinite limits of neural networks
por: Lauditi, Clarissa, et al.
Publicado: (2025)

Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks
por: Nguyen, Minh-Toan, et al.
Publicado: (2026)

Daydreaming Hopfield Networks and their surprising effectiveness on correlated data
por: Serricchio, Ludovica, et al.
Publicado: (2024)

Universal characteristics of deep neural network loss surfaces from random matrix theory
por: Baskerville, Nicholas P, et al.
Publicado: (2022)

Applying statistical learning theory to deep learning
por: Gerbelot, Cédric, et al.
Publicado: (2023)

Storage capacity of perceptron with variable selection
por: Xu, Yingying, et al.
Publicado: (2025)

Critical feature learning in deep neural networks
por: Fischer, Kirsten, et al.
Publicado: (2024)

On the origin of neural scaling laws: from random graphs to natural language
por: Barkeshli, Maissam, et al.
Publicado: (2026)

Fundamental limits of learning in sequence multi-index models and deep attention networks: High-dimensional asymptotics and sharp thresholds
por: Troiani, Emanuele, et al.
Publicado: (2025)

Dimension-free deterministic equivalents and scaling laws for random feature regression
por: Defilippis, Leonardo, et al.
Publicado: (2024)

A universal compression theory for lottery ticket hypothesis and neural scaling laws
por: Wang, Hong-Yi, et al.
Publicado: (2025)

Supervised and Unsupervised protocols for hetero-associative neural networks
por: Alessandrelli, Andrea, et al.
Publicado: (2025)

Deep neural networks from the perspective of ergodic theory
por: Zhang, Fan
Publicado: (2023)

An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem
por: Nam, Yoonsoo, et al.
Publicado: (2024)

Training neural networks with structured noise improves classification and generalization
por: Benedetti, Marco, et al.
Publicado: (2023)

Exact full-RSB SAT/UNSAT transition in infinitely wide two-layer neural networks
por: Annesi, Brandon L., et al.
Publicado: (2024)

Coding schemes in neural networks learning classification tasks
por: van Meegen, Alexander, et al.
Publicado: (2024)

How does training shape the Riemannian geometry of neural network representations?
por: Zavatone-Veth, Jacob A., et al.
Publicado: (2023)

Boundary between noise and information applied to filtering neural network weight matrices
por: Staats, Max, et al.
Publicado: (2022)

Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysis
por: Lee, Donghee, et al.
Publicado: (2024)

The twin peaks of learning neural networks
por: Demyanenko, Elizaveta, et al.
Publicado: (2024)

Random Features Hopfield Networks generalize retrieval to previously unseen examples
por: Kalaj, Silvio, et al.
Publicado: (2024)

Overparametrization bends the landscape: BBP transitions at initialization in simple Neural Networks
por: Annesi, Brandon Livio, et al.
Publicado: (2025)

Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block model
por: Duranthon, O., et al.
Publicado: (2025)

Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activations
por: Baldassi, Carlo, et al.
Publicado: (2019)