Cover Image

Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mohamadi, Mohamad Amin, Li, Zhiyuan, Wu, Lei, Sutherland, Danica J.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

To Grok Grokking: Provable Grokking in Ridge Regression
by: Xu, Mingyue, et al.
Published: (2026)

Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity
by: Xie, Shuo, et al.
Published: (2024)

Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation
by: Mohamadi, Mohamad Amin, et al.
Published: (2025)

GrokAlign: Geometric Characterisation and Acceleration of Grokking
by: Walker, Thomas, et al.
Published: (2025)

Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Model
by: Xu, Zhiwei, et al.
Published: (2025)

NeuralGrok: Accelerate Grokking by Neural Gradient Transformation
by: Zhou, Xinyu, et al.
Published: (2025)

Grokking of Diffusion Models: Case Study on Modular Addition
by: Kim, Joon Hyeok, et al.
Published: (2026)

Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding
by: Levi, Noam, et al.
Published: (2023)

On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
by: He, Jianliang, et al.
Published: (2026)

Provable Benefits of Sinusoidal Activation for Modular Addition
by: Huang, Tianlong, et al.
Published: (2025)

Deep Networks Always Grok and Here is Why
by: Humayun, Ahmed Imtiaz, et al.
Published: (2024)

Grokking Modular Polynomials
by: Doshi, Darshil, et al.
Published: (2024)

Beyond Progress Measures: Theoretical Insights into the Mechanism of Grokking
by: Gu, Zihan, et al.
Published: (2025)

Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics
by: Ren, Yi, et al.
Published: (2024)

Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex
by: Kumar, Tanishq, et al.
Published: (2024)

Why Do More Experts Fail? A Theoretical Analysis of Model Merging
by: Wang, Zijing, et al.
Published: (2025)

Learning Dynamics of LLM Finetuning
by: Ren, Yi, et al.
Published: (2024)

Generalized Coverage for More Robust Low-Budget Active Learning
by: Bae, Wonho, et al.
Published: (2024)

Learning Representations for Independence Testing
by: Xu, Nathaniel, et al.
Published: (2024)

Efficient kernelized bandit algorithms via exploration distributions
by: Hu, Bingshan, et al.
Published: (2025)

Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling
by: Bae, Wonho, et al.
Published: (2023)

A Theoretical Framework for Grokking: Interpolation followed by Riemannian Norm Minimisation
by: Boursier, Etienne, et al.
Published: (2025)

Uncertainty Herding: One Active Learning Method for All Label Budgets
by: Bae, Wonho, et al.
Published: (2024)

Grokked Models are Better Unlearners
by: Liang, Yuanbang, et al.
Published: (2025)

Latent Algorithmic Structure Precedes Grokking: A Mechanistic Study of ReLU MLPs on Modular Arithmetic
by: Swaroop, Anand
Published: (2026)

Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomials
by: Furuta, Hiroki, et al.
Published: (2024)

Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition
by: Clauw, Kenzo, et al.
Published: (2024)

Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition
by: Huang, Yufei, et al.
Published: (2024)

Language Models "Grok" to Copy
by: Lv, Ang, et al.
Published: (2024)

Maximum Mean Discrepancy with Unequal Sample Sizes via Generalized U-Statistics
by: Wei, Aaron, et al.
Published: (2025)

The Complexity Dynamics of Grokking
by: DeMoss, Branton, et al.
Published: (2024)

Measuring Sharpness in Grokking
by: Miller, Jack, et al.
Published: (2024)

Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks
by: Minegishi, Gouki, et al.
Published: (2023)

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking
by: Lyu, Kaifeng, et al.
Published: (2023)

Kernel Model Validation: How To Do It, And Why You Should Care
by: Graziani, Carlo, et al.
Published: (2025)

Practical Kernel Tests of Conditional Independence
by: Pogodin, Roman, et al.
Published: (2024)

Survival of the Fittest Representation: A Case Study with Modular Addition
by: Ding, Xiaoman Delores, et al.
Published: (2024)

Why Do We Need Warm-up? A Theoretical Perspective
by: Alimisis, Foivos, et al.
Published: (2025)

A Theoretical Analysis of Noise Geometry in Stochastic Gradient Descent
by: Wang, Mingze, et al.
Published: (2023)

GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers
by: Ai, Guoguo, et al.
Published: (2024)