Cover Image

A universal compression theory for lottery ticket hypothesis and neural scaling laws

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Hong-Yi, Luo, Di, Poggio, Tomaso, Chuang, Isaac L., Ziyin, Liu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Disordered Systems and Neural Networks Information Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Formation of Representations in Neural Networks
by: Ziyin, Liu, et al.
Published: (2024)

Heterosynaptic Circuits Are Universal Gradient Machines
by: Ziyin, Liu, et al.
Published: (2025)

Parameter Symmetry Potentially Unifies Deep Learning Theory
by: Ziyin, Liu, et al.
Published: (2025)

Proof of a perfect platonic representation hypothesis
by: Ziyin, Liu, et al.
Published: (2025)

The phase diagram of compressed sensing with $\ell_0$-norm regularization
by: Barbier, Damien, et al.
Published: (2024)

Phase transition in compressed sensing using log-sum penalty and adaptive smoothing
by: Morita, Keisuke, et al.
Published: (2026)

Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks
by: Nguyen, Minh-Toan, et al.
Published: (2026)

Renormalization group for deep neural networks: Universality of learning and scaling laws
by: Coppola, Gorka Peraza, et al.
Published: (2025)

Dynamics of neural scaling laws in random feature regression with powerlaw-distributed kernel eigenvalues
by: Kramp, Jakob, et al.
Published: (2026)

Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting
by: Takahashi, Takashi, et al.
Published: (2020)

Statistical mechanics of the maximum-average submatrix problem
by: Erba, Vittorio, et al.
Published: (2023)

The Exponential Capacity of Dense Associative Memories
by: Lucibello, Carlo, et al.
Published: (2023)

BBP transition and the leading eigenvector of the spiked Wigner model with inhomogeneous noise
by: Ferreira, Leonardo S., et al.
Published: (2026)

Geometry and universal scaling of Pareto-optimal signal compression
by: Berx, Jonas
Published: (2025)

Fixed width treelike neural networks capacity analysis -- generic activations
by: Stojnic, Mihailo
Published: (2024)

Exact capacity of the \emph{wide} hidden layer treelike neural networks with generic activations
by: Stojnic, Mihailo
Published: (2024)

Bayes-optimal learning of an extensive-width neural network from quadratically many samples
by: Maillard, Antoine, et al.
Published: (2024)

The maximum-average subtensor problem: equilibrium and out-of-equilibrium properties
by: Erba, Vittorio, et al.
Published: (2025)

Role of Bootstrap Averaging in Generalized Approximate Message Passing
by: Takahashi, Takashi
Published: (2023)

Compressed sensing with l0-norm: statistical physics analysis and algorithms for signal recovery
by: Barbier, D., et al.
Published: (2023)

Arbitrage equilibrium and the emergence of universal microstructure in deep neural networks
by: Venkatasubramanian, Venkat, et al.
Published: (2024)

Statistical mechanics of extensive-width Bayesian neural networks near interpolation
by: Barbier, Jean, et al.
Published: (2025)

Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation
by: Barbier, Jean, et al.
Published: (2025)

Implicit bias produces neural scaling laws in learning curves, from perceptrons to deep networks
by: D'Amico, Francesco, et al.
Published: (2025)

When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study
by: Obuchi, Tomoyuki, et al.
Published: (2024)

Rare dense solutions clusters in asymmetric binary perceptrons -- local entropy via fully lifted RDT
by: Stojnic, Mihailo
Published: (2025)

Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
by: Boncoraglio, Fabrizio, et al.
Published: (2025)

Bayes optimal learning of attention-indexed models
by: Boncoraglio, Fabrizio, et al.
Published: (2025)

Storage capacity of perceptron with variable selection
by: Xu, Yingying, et al.
Published: (2025)

The Random Subsequence Model and Uniform Codes for the Deletion Channel
by: Jeong, Ryan, et al.
Published: (2026)

Deep ReLU networks -- injectivity capacity upper bounds
by: Stojnic, Mihailo
Published: (2024)

Injectivity capacity of ReLU gates
by: Stojnic, Mihailo
Published: (2024)

Photonic Simulation of Localization Phenomena Using Boson Sampling
by: Kulkarni, Anuprita V., et al.
Published: (2024)

The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks
by: Erba, Vittorio, et al.
Published: (2025)

On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariance
by: Barbier, Jean, et al.
Published: (2024)

The replica-symmetric free energy for Ising spin glasses with orthogonally invariant couplings
by: Fan, Zhou, et al.
Published: (2021)

Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
by: Ziyin, Liu, et al.
Published: (2025)

Graphicality of power-law and double power-law degree sequences
by: Valigi, Pietro, et al.
Published: (2025)

Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime
by: Camilli, Francesco, et al.
Published: (2025)

On the origin of neural scaling laws: from random graphs to natural language
by: Barkeshli, Maissam, et al.
Published: (2026)