Image de couverture de livre

Don't be lazy: CompleteP enables compute-efficient deep transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dey, Nolan, Zhang, Bin Claire, Noci, Lorenzo, Li, Mufan, Bordelon, Blake, Bergsma, Shane, Pehlevan, Cengiz, Hanin, Boris, Hestness, Joel
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Sparse maximal update parameterization: A holistic approach to sparse training dynamics
par: Dey, Nolan, et autres
Publié: (2024)

Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs
par: Bergsma, Shane, et autres
Publié: (2025)

Hyperparameter Transfer with Mixture-of-Expert Layers
par: Jiang, Tianze, et autres
Publié: (2026)

Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learning
par: Bordelon, Blake, et autres
Publié: (2026)

Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
par: Bordelon, Blake, et autres
Publié: (2025)

Scaling with Collapse: Efficient and Predictable Training of LLM Families
par: Bergsma, Shane, et autres
Publié: (2025)

How Feature Learning Can Improve Neural Scaling Laws
par: Bordelon, Blake, et autres
Publié: (2024)

Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
par: Lauditi, Clarissa, et autres
Publié: (2026)

Adaptive kernel predictors from feature-learning infinite limits of neural networks
par: Lauditi, Clarissa, et autres
Publié: (2025)

Transfer Learning in Infinite Width Feature Learning Networks
par: Lauditi, Clarissa, et autres
Publié: (2025)

A Dynamical Model of Neural Scaling Laws
par: Bordelon, Blake, et autres
Publié: (2024)

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
par: Bergsma, Shane, et autres
Publié: (2025)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
par: Bergsma, Shane, et autres
Publié: (2025)

Infinite Limits of Multi-head Transformer Dynamics
par: Bordelon, Blake, et autres
Publié: (2024)

Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
par: Bordelon, Blake, et autres
Publié: (2025)

Summary statistics of learning link changing neural representations to behavior
par: Zavatone-Veth, Jacob A., et autres
Publié: (2025)

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers
par: Gray, Gavia, et autres
Publié: (2024)

Grokking as the Transition from Lazy to Rich Training Dynamics
par: Kumar, Tanishq, et autres
Publié: (2023)

Dynamically Learning to Integrate in Recurrent Neural Networks
par: Bordelon, Blake, et autres
Publié: (2025)

Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex
par: Kumar, Tanishq, et autres
Publié: (2024)

Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
par: Atanasov, Alexander, et autres
Publié: (2025)

Learning Curves for Noisy Heterogeneous Feature-Subsampled Ridge Ensembles
par: Ruben, Benjamin S., et autres
Publié: (2023)

Discovering alternative solutions beyond the simplicity bias in recurrent neural networks
par: Qian, William, et autres
Publié: (2025)

Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling
par: Halder, Indranil, et autres
Publié: (2025)

An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models
par: Wang, Binxu, et autres
Publié: (2025)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
par: Dong, Yihe, et autres
Publié: (2025)

Scaling Laws for Precision
par: Kumar, Tanishq, et autres
Publié: (2024)

Learning richness modulates equality reasoning in neural networks
par: Tong, William L., et autres
Publié: (2025)

MLPs Learn In-Context on Regression and Classification Tasks
par: Tong, William L., et autres
Publié: (2024)

Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Model
par: Bordelon, Blake, et autres
Publié: (2026)

Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism
par: Erdogan, Mete, et autres
Publié: (2025)

Nadaraya-Watson kernel smoothing as a random energy model
par: Zavatone-Veth, Jacob A., et autres
Publié: (2024)

Convex Relaxation for Solving Large-Margin Classifiers in Hyperbolic Space
par: Yang, Sheng, et autres
Publié: (2024)

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover
par: Halder, Indranil, et autres
Publié: (2026)

Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness
par: Attias, Elie, et autres
Publié: (2024)

A note on the dynamics of extended-context disordered kinetic spin models
par: Zavatone-Veth, Jacob A., et autres
Publié: (2025)

Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry
par: Bozkurt, Bariscan, et autres
Publié: (2023)

A solvable model of learning generative diffusion: theory and insights
par: Cui, Hugo, et autres
Publié: (2025)

A Random Matrix Theory Perspective on the Consistency of Diffusion Models
par: Wang, Binxu, et autres
Publié: (2026)

Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Traces
par: Tong, William L., et autres
Publié: (2026)