Cover Image

Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kunin, Daniel, Raventós, Allan, Dominé, Clémentine, Chen, Feng, Klindt, David, Saxe, Andrew, Ganguli, Surya
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Deriving Neural Scaling Laws from the statistics of natural language
by: Cagnetta, Francesco, et al.
Published: (2026)

Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks
by: Chen, Feng, et al.
Published: (2023)

Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning
by: Chen, Feng, et al.
Published: (2025)

Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing
by: Njaradi, Valentina, et al.
Published: (2026)

From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks
by: Dominé, Clémentine C. J., et al.
Published: (2024)

A Theory of Initialisation's Impact on Specialisation
by: Jarvis, Devon, et al.
Published: (2025)

A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning
by: Anguita, Nicolas, et al.
Published: (2026)

An analytic theory of creativity in convolutional diffusion models
by: Kamb, Mason, et al.
Published: (2024)

Fooling LLM graders into giving better grades through neural activity guided adversarial prompting
by: Yamamura, Atsushi, et al.
Published: (2024)

Features are fate: a theory of transfer learning in high-dimensional regression
by: Tahir, Javan, et al.
Published: (2024)

Early learning of the optimal constant solution in neural networks and humans
by: Rubruck, Jirko, et al.
Published: (2024)

Error‐controlled feature selection for ultrahigh‐dimensional and highly correlated feature space using deep learning
by: Arkaprabha Ganguli, et al.
Published: (2024)

Superposition disentanglement of neural representations reveals hidden alignment
by: Longon, André, et al.
Published: (2025)

Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks
by: Kunin, Daniel, et al.
Published: (2025)

The geometry and dynamics of annealed optimization in the coherent Ising machine with hidden and planted solutions
by: Ghimenti, Federico, et al.
Published: (2025)

Propiedad, libertad republicana y Renta Básica de Ciudadanía
by: Daniel Raventós
Published: (2005)

Solving adversarial examples requires solving exponential misalignment
by: Salvatore, Alessandro, et al.
Published: (2026)

Lorentz-boosted diffusion: initial value formulation and exact solutions
by: Gavassino, Lorenzo
Published: (2026)

Machine learning reveals features of spinon Fermi surface
by: Zhang, Kevin, et al.
Published: (2023)

Short-term plasticity recalls forgotten memories through a trampoline mechanism
by: Del Gaudio, Martina, et al.
Published: (2025)

APUNTES SOBRE EL RAP POLÍTICO BOLIVIANO
by: Johana Kunin
Published: (2009)

Percepción social del riesgo y dinâmicas de género en la producción agrícola basada en plaguicidas en la pampa húmeda Argentina
by: Johana Kunin
Published: (2020)

Reseña de "Ciberculturas juveniles. Los jóvenes, sus prácticas y sus representaciones en la era de Internet" de Marcelo Urresti (edit.)
by: Johana Kunin
Published: (2008)

Pariendo madres: talleres de parto y de crianza en un distrito rural bonaerense
by: Johana Kunin
Published: (2021)

Los "medio putos": masculinidades subalternas y dinámicas de género alternativas en la rural Pampa húmeda argentina (2014-2017)
by: Johana Kunin
Published: (2021)

Cuando los paradigmas agroecológicos y del parto respetado no encuentran las respuestas esperadas
by: Johana Kunin
Published: (2022)

“Acá se sabe si la gente se aísla”: (anti)anonimato, cuidado y poder en localidades medianas y pequeñas en tiempos de COVID-19
by: Johana Kunin
Published: (2021)

What is the $\textit{intrinsic}$ dimension of your binary data? -- and how to compute it quickly
by: Hanika, Tom, et al.
Published: (2024)

Contrastive Concept-Tree Search for LLM-Assisted Algorithm Discovery
by: Leleu, Timothee, et al.
Published: (2026)

From Kepler to Newton: Inductive Biases Guide Learned World Models in Transformers
by: Liu, Ziming, et al.
Published: (2026)

Reshaping Global Loop Structure to Accelerate Local Optimization by Smoothing Rugged Landscapes
by: Leleu, Timothee, et al.
Published: (2026)

Data Whitening Improves Sparse Autoencoder Learning
by: Saraswatula, Ashwin, et al.
Published: (2025)

Sea-bed photographs (benthos) along ROV profile PS12/305
by: Gutt, Julian, et al.
Published: (2013)

Sea-bed photographs (benthos) along ROV profile PS12/308
by: Gutt, Julian, et al.
Published: (2013)

Sea-floor videos and photographs (benthos) along 16 ROV profiles during POLARSTERN cruise ANT-VI/3, Weddell Sea
by: Gutt, Julian, et al.
Published: (2011)

Softmax $\geq$ Linear: Transformers may learn to classify in-context by kernel gradient descent
by: Dragutinović, Sara, et al.
Published: (2025)

Exact values for unbalanced Zarankiewicz numbers
by: Chen, Guangzhou, et al.
Published: (2022)

Large analysis of genetic manipulations reveals an inverse correlation between initial alcohol resistance and rapid tolerance phenotypes
by: Maggie M. Chvilicek, et al.
Published: (2024)

Lo que fue ya no es y lo nuevo aún no toma forma: elecciones 2006 en perspectiva histórica
by: Ciska Raventós
Published: (2008)

Combo del ICE en el momento culminante De las protestas Sondeo telefónico 24-25 de marzo del 2000
by: Ciska Raventós
Published: (2004)