Image de couverture de livre

Deep Grokking: Would Deep Neural Networks Generalize Better?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Fan, Simin, Pascanu, Razvan, Jaggi, Martin
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

NeuralGrok: Accelerate Grokking by Neural Gradient Transformation
par: Zhou, Xinyu, et autres
Publié: (2025)

What Can Grokking Teach Us About Learning Under Nonstationarity?
par: Lyle, Clare, et autres
Publié: (2025)

DoGE: Domain Reweighting with Generalization Estimation
par: Fan, Simin, et autres
Publié: (2023)

NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagation
par: Li, Qinyu, et autres
Publié: (2025)

HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimation
par: Zhou, Xinyu, et autres
Publié: (2024)

Layerwise LQR for Geometry-Aware Optimization of Deep Networks
par: Dufort-Labbé, Simon, et autres
Publié: (2026)

Latent Space Representations of Neural Algorithmic Reasoners
par: Mirjanić, Vladimir V., et autres
Publié: (2023)

GRAPE: Optimize Data Mixture for Group Robust Multi-target Adaptive Pretraining
par: Fan, Simin, et autres
Publié: (2025)

Grokked Models are Better Unlearners
par: Liang, Yuanbang, et autres
Publié: (2025)

Disentangling the Causes of Plasticity Loss in Neural Networks
par: Lyle, Clare, et autres
Publié: (2024)

Lattice: Learning to Efficiently Compress the Memory
par: Karami, Mahdi, et autres
Publié: (2025)

Meta-learning how to Share Credit among Macro-Actions
par: Hosu, Ionel-Alexandru, et autres
Publié: (2025)

Revisiting Adam for Streaming Reinforcement Learning
par: Gogianu, Florin, et autres
Publié: (2026)

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
par: Kosson, Atli, et autres
Publié: (2023)

Grokking Beyond Neural Networks: An Empirical Exploration with Model Complexity
par: Miller, Jack, et autres
Publié: (2023)

Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
par: Galashov, Alexandre, et autres
Publié: (2024)

Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking
par: Han, Ting, et autres
Publié: (2025)

Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks
par: Minegishi, Gouki, et autres
Publié: (2023)

Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test
par: Li, Ziyue, et autres
Publié: (2025)

Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization
par: Masarczyk, Wojciech, et autres
Publié: (2025)

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO
par: Moalla, Skander, et autres
Publié: (2024)

Attention as a Hypernetwork
par: Schug, Simon, et autres
Publié: (2024)

MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling
par: Karami, Mahdi, et autres
Publié: (2025)

Softmax is not Enough (for Sharp Size Generalisation)
par: Veličković, Petar, et autres
Publié: (2024)

To Grok Grokking: Provable Grokking in Ridge Regression
par: Xu, Mingyue, et autres
Publié: (2026)

Hadamard product in deep learning: Introduction, Advances and Challenges
par: Chrysos, Grigorios G, et autres
Publié: (2025)

Late-Stage Generalization Collapse in Grokking: Detecting anti-grokking with Weightwatcher
par: Prakash, Hari K, et autres
Publié: (2026)

Round and Round We Go! What makes Rotary Positional Encodings useful?
par: Barbero, Federico, et autres
Publié: (2024)

Improving Generalization of Deep Neural Networks by Optimum Shifting
par: Zhou, Yuyan, et autres
Publié: (2024)

Grokking and Generalization Collapse: Insights from \texttt{HTSR} theory
par: Prakash, Hari K., et autres
Publié: (2025)

Transformers meet Neural Algorithmic Reasoners
par: Bounsi, Wilfried, et autres
Publié: (2024)

Explaining Grokking and Information Bottleneck through Neural Collapse Emergence
par: Sakamoto, Keitaro, et autres
Publié: (2025)

Personalized Collaborative Fine-Tuning for On-Device Large Language Models
par: Wagner, Nicolas, et autres
Publié: (2024)

State Soup: In-Context Skill Learning, Retrieval and Mixing
par: Pióro, Maciej, et autres
Publié: (2024)

Grokking as Dimensional Phase Transition in Neural Networks
par: Wang, Ping
Publié: (2026)

LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities
par: Schmied, Thomas, et autres
Publié: (2025)

The Complexity Dynamics of Grokking
par: DeMoss, Branton, et autres
Publié: (2024)

Measuring Sharpness in Grokking
par: Miller, Jack, et autres
Publié: (2024)

Grokking Group Multiplication with Cosets
par: Stander, Dashiell, et autres
Publié: (2023)

Perplexity Cannot Always Tell Right from Wrong
par: Veličković, Petar, et autres
Publié: (2026)