Cover Image

Grokking as Structural Inference: Transformers Need Bayesian Lottery Tickets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hidajat, Kai, Stoll, Solden, An, Joseph
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Sparsity of the Strong Lottery Ticket Hypothesis
by: Natale, Emanuele, et al.
Published: (2024)

Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks
by: Minegishi, Gouki, et al.
Published: (2023)

KS-Lottery: Finding Certified Lottery Tickets for Multilingual Language Models
by: Yuan, Fei, et al.
Published: (2024)

The Strong Lottery Ticket Hypothesis for Multi-Head Attention Mechanisms
by: Otsuka, Hikari, et al.
Published: (2025)

Uncovering a Winning Lottery Ticket with Continuously Relaxed Bernoulli Gates
by: Tsayag, Itamar, et al.
Published: (2026)

Partially Frozen Random Networks Contain Compact Strong Lottery Tickets
by: Otsuka, Hikari, et al.
Published: (2024)

Investigating the Lottery Ticket Hypothesis for Variational Quantum Circuits
by: Kölle, Michael, et al.
Published: (2025)

LOTUS: Improving Transformer Efficiency with Sparsity Pruning and Data Lottery Tickets
by: Upadhyay, Ojasw
Published: (2024)

A Neural Scaling Law from Lottery Ticket Ensembling
by: Liu, Ziming, et al.
Published: (2023)

On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
by: He, Jianliang, et al.
Published: (2026)

Bayesian Lottery Ticket Hypothesis
by: Kuhn, Nicholas, et al.
Published: (2026)

Winning the Lottery by Preserving Network Training Dynamics with Concrete Ticket Search
by: Arora, Tanay, et al.
Published: (2025)

NeuralGrok: Accelerate Grokking by Neural Gradient Transformation
by: Zhou, Xinyu, et al.
Published: (2025)

Find A Winning Sign: Sign Is All We Need to Win the Lottery
by: Oh, Junghun, et al.
Published: (2025)

Topological Signatures of Grokking
by: Tang, Yifan, et al.
Published: (2026)

Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomials
by: Furuta, Hiroki, et al.
Published: (2024)

Martingale Neural Operators: Learning Stochastic Marginals via Doob-Meyer Factorization
by: Hidajat, Kai
Published: (2026)

Grokking in Linear Models for Logistic Regression
by: Das, Nataraj, et al.
Published: (2026)

Grokking Explained: A Statistical Phenomenon
by: Carvalho, Breno W., et al.
Published: (2025)

Controlling Grokking with Nonlinearity and Data Symmetry
by: Salah, Ahmed, et al.
Published: (2024)

The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure
by: Xu, Yongzhong
Published: (2026)

Grokfast: Accelerated Grokking by Amplifying Slow Gradients
by: Lee, Jaerin, et al.
Published: (2024)

Understanding Grokking Through A Robustness Viewpoint
by: Tan, Zhiquan, et al.
Published: (2023)

Progress Measures for Grokking on Real-world Tasks
by: Golechha, Satvik
Published: (2024)

Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
by: Verma, Lucky
Published: (2026)

Low-Dimensional and Transversely Curved Optimization Dynamics in Grokking
by: Xu, Yongzhong
Published: (2026)

Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation
by: Salah, Ahmed, et al.
Published: (2025)

The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold
by: Musat, Tiberiu
Published: (2025)

Feature Lottery? A Bifurcation Theory of Concept Emergence
by: Yang, Fuming
Published: (2026)

Grokking Finite-Dimensional Algebra
by: Notsawo, Pascal Jr Tikeng, et al.
Published: (2026)

Grokking Group Multiplication with Cosets
by: Stander, Dashiell, et al.
Published: (2023)

Muon Optimizer Accelerates Grokking
by: Tveit, Amund, et al.
Published: (2025)

The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference
by: Qasim, Kaleem Ullah, et al.
Published: (2026)

Early-Warning Signals of Grokking via Loss-Landscape Geometry
by: Xu, Yongzhong
Published: (2026)

Grokking and Generalization Collapse: Insights from \texttt{HTSR} theory
by: Prakash, Hari K., et al.
Published: (2025)

Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks
by: Fracastoro, Giulia, et al.
Published: (2025)

The Multiple Ticket Hypothesis: Random Sparse Subnetworks Suffice for RLVR
by: Adewuyi, Israel, et al.
Published: (2026)

Jackpot! Alignment as a Maximal Lottery
by: Maura-Rivero, Roberto-Rafael, et al.
Published: (2025)

Acceleration of Grokking in Learning Arithmetic Operations via Kolmogorov-Arnold Representation
by: Park, Yeachan, et al.
Published: (2024)

Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking
by: Tian, Yuandong
Published: (2025)