Copertina

Learning curves theory for hierarchically compositional data with power-law distributed features

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cagnetta, Francesco, Kang, Hyunmo, Wyart, Matthieu
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Disordered Systems and Neural Networks
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Towards a theory of how the structure of language is acquired by deep neural networks
di: Cagnetta, Francesco, et al.
Pubblicazione: (2024)

Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures
di: Cagnetta, Francesco, et al.
Pubblicazione: (2025)

Deep networks learn to parse uniform-depth context-free languages from local statistics
di: Parley, Jack T., et al.
Pubblicazione: (2026)

Sampling Data with Chains of Forward-Backward Diffusion Steps
di: Kang, Hyunmo, et al.
Pubblicazione: (2026)

How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model
di: Tomasini, Umberto, et al.
Pubblicazione: (2024)

On the different regimes of Stochastic Gradient Descent
di: Sclocchi, Antonio, et al.
Pubblicazione: (2023)

Probing the Latent Hierarchical Structure of Data via Diffusion Models
di: Sclocchi, Antonio, et al.
Pubblicazione: (2024)

Spectral Analysis of Representational Similarity with Limited Neurons
di: Kang, Hyunmo, et al.
Pubblicazione: (2025)

Implicit bias produces neural scaling laws in learning curves, from perceptrons to deep networks
di: D'Amico, Francesco, et al.
Pubblicazione: (2025)

On the Emergence of Linear Analogies in Word Embeddings
di: Korchinski, Daniel J., et al.
Pubblicazione: (2025)

A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data
di: Sclocchi, Antonio, et al.
Pubblicazione: (2024)

Dimension-free deterministic equivalents and scaling laws for random feature regression
di: Defilippis, Leonardo, et al.
Pubblicazione: (2024)

Symmetry in language statistics shapes the geometry of model representations
di: Karkada, Dhruva, et al.
Pubblicazione: (2026)

A unified theory of feature learning in RNNs and DNNs
di: Bauer, Jan P., et al.
Pubblicazione: (2026)

Random features and polynomial rules
di: Aguirre-López, Fabián, et al.
Pubblicazione: (2024)

Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Model
di: Bordelon, Blake, et al.
Pubblicazione: (2026)

Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting
di: Thériault, Robin, et al.
Pubblicazione: (2024)

An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem
di: Nam, Yoonsoo, et al.
Pubblicazione: (2024)

Graphicality of power-law and double power-law degree sequences
di: Valigi, Pietro, et al.
Pubblicazione: (2025)

Adaptive kernel predictors from feature-learning infinite limits of neural networks
di: Lauditi, Clarissa, et al.
Pubblicazione: (2025)

Asymptotics of feature learning in two-layer networks after one gradient-step
di: Cui, Hugo, et al.
Pubblicazione: (2024)

Spring-block theory of feature learning in deep neural networks
di: Shi, Cheng, et al.
Pubblicazione: (2024)

Preferential attachment and power-law degree distributions in heterogeneous multilayer hypergraphs
di: Di Lauro, Francesco, et al.
Pubblicazione: (2025)

Dynamics of neural scaling laws in random feature regression with powerlaw-distributed kernel eigenvalues
di: Kramp, Jakob, et al.
Pubblicazione: (2026)

A universal compression theory for lottery ticket hypothesis and neural scaling laws
di: Wang, Hong-Yi, et al.
Pubblicazione: (2025)

Analytic theory of dropout regularization
di: Mori, Francesco, et al.
Pubblicazione: (2025)

Asymptotic theory of in-context learning by linear attention
di: Lu, Yue M., et al.
Pubblicazione: (2024)

Applying statistical learning theory to deep learning
di: Gerbelot, Cédric, et al.
Pubblicazione: (2023)

Deep neural networks from the perspective of ergodic theory
di: Zhang, Fan
Pubblicazione: (2023)

Field theory for optimal signal propagation in ResNets
di: Fischer, Kirsten, et al.
Pubblicazione: (2023)

Self-attention as an attractor network: transient memories without backpropagation
di: D'Amico, Francesco, et al.
Pubblicazione: (2024)

A solvable model of learning generative diffusion: theory and insights
di: Cui, Hugo, et al.
Pubblicazione: (2025)

Supervised Hebbian Learning
di: Alemanno, Francesco, et al.
Pubblicazione: (2022)

A Random-Matrix Criterion for Initializing Gated Recurrent Neural Networks
di: Fioratti, Tommaso, et al.
Pubblicazione: (2026)

Short-range depinning in the presence of velocity-weakening
di: de Geus, Tom W. J., et al.
Pubblicazione: (2024)

Dynamical heterogeneities of thermal creep in pinned interfaces
di: de Geus, Tom W. J., et al.
Pubblicazione: (2024)

Regularization, early-stopping and dreaming: a Hopfield-like setup to address generalization and overfitting
di: Agliari, Elena, et al.
Pubblicazione: (2023)

Where You Place the Norm Matters: From Prejudiced to Neutral Initializations
di: Francazi, Emanuele, et al.
Pubblicazione: (2025)

Transfer Learning in Infinite Width Feature Learning Networks
di: Lauditi, Clarissa, et al.
Pubblicazione: (2025)

Physical Reinforcement Learning
di: Dillavou, Sam, et al.
Pubblicazione: (2025)