Cover Image

Latent Algorithmic Structure Precedes Grokking: A Mechanistic Study of ReLU MLPs on Modular Arithmetic

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Swaroop, Anand
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Looped ReLU MLPs May Be All You Need as Practical Programmable Computers
by: Liang, Yingyu, et al.
Published: (2024)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
by: Manik, Md Motaleb Hossen, et al.
Published: (2025)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

The Resurrection of the ReLU
by: Horuz, Coşku Can, et al.
Published: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Complexity of One-Dimensional ReLU DNNs
by: Kogan, Jonathan, et al.
Published: (2025)

Symmetric Matrix Completion with ReLU Sampling
by: Liu, Huikang, et al.
Published: (2024)

An Accelerated Alternating Partial Bregman Algorithm for ReLU-based Matrix Decomposition
by: Wang, Qingsong, et al.
Published: (2025)

An Efficient Alternating Algorithm for ReLU-based Symmetric Matrix Decomposition
by: Wang, Qingsong
Published: (2025)

ReCA: A Parametric ReLU Composite Activation Function
by: Chidiac, John, et al.
Published: (2025)

Is ReLU Adversarially Robust?
by: Sooksatra, Korn, et al.
Published: (2024)

SurvReLU: Inherently Interpretable Survival Analysis via Deep ReLU Networks
by: Sun, Xiaotong, et al.
Published: (2024)

Online Realizable Regression and Applications for ReLU Networks
by: Doron-Arad, Ilan, et al.
Published: (2026)

Constraining the outputs of ReLU neural networks
by: Alexandr, Yulia, et al.
Published: (2025)

Toric geometry of ReLU neural networks
by: Fu, Yaoying
Published: (2025)

Pathwise Explanation of ReLU Neural Networks
by: Lim, Seongwoo, et al.
Published: (2025)

Nearly Optimal Differentially Private ReLU Regression
by: Ding, Meng, et al.
Published: (2025)

Optimal Sets and Solution Paths of ReLU Networks
by: Mishkin, Aaron, et al.
Published: (2023)

On Size-Independent Sample Complexity of ReLU Networks
by: Sellke, Mark
Published: (2023)

Convexity in ReLU Neural Networks: beyond ICNNs?
by: Gagneux, Anne, et al.
Published: (2025)

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU
by: Qiu, Qi, et al.
Published: (2024)

A Momentum Accelerated Algorithm for ReLU-based Nonlinear Matrix Decomposition
by: Wang, Qingsong, et al.
Published: (2024)

Brownian ReLU(Br-ReLU): A New Activation Function for a Long-Short Term Memory (LSTM) Network
by: Awiakye-Marfo, George, et al.
Published: (2026)

Schauder Bases for $C[0, 1]$ Using ReLU, Softplus and Two Sigmoidal Functions
by: Ganesh, Anand, et al.
Published: (2025)

Bridging the Gap Between Approximation and Learning via Optimal Approximation by ReLU MLPs of Maximal Regularity
by: Hong, Ruiyang, et al.
Published: (2024)

Stochastic Bandits with ReLU Neural Networks
by: Xu, Kan, et al.
Published: (2024)

On Space Folds of ReLU Neural Networks
by: Lewandowski, Michal, et al.
Published: (2025)

Hedonic Neurons: A Mechanistic Mapping of Latent Coalitions in Transformer MLPs
by: Chowdhury, Tanya, et al.
Published: (2025)

Injectivity capacity of ReLU gates
by: Stojnic, Mihailo
Published: (2024)

ReLU integral probability metric and its applications
by: Park, Yuha, et al.
Published: (2025)

Stably unactivated neurons in ReLU neural networks
by: Brownlowe, Natalie, et al.
Published: (2024)

Topological Expressivity of ReLU Neural Networks
by: Ergen, Ekin, et al.
Published: (2023)

Three Quantization Regimes for ReLU Networks
by: Ou, Weigutian, et al.
Published: (2024)

On the failure of ReLU activation for physics-informed machine learning
by: Rowan, Conor
Published: (2025)

Injectivity of ReLU-layers: Tools from Frame Theory
by: Haider, Daniel, et al.
Published: (2024)

On the Local Complexity of Linear Regions in Deep ReLU Networks
by: Patel, Niket, et al.
Published: (2024)

ReLU Networks as Random Functions: Their Distribution in Probability Space
by: Chaudhari, Shreyas, et al.
Published: (2025)

Noisy Interpolation Learning with Shallow Univariate ReLU Networks
by: Joshi, Nirmit, et al.
Published: (2023)

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks
by: Vallin, Jonatan, et al.
Published: (2024)

Training a Two Layer ReLU Network Analytically
by: Barbu, Adrian
Published: (2023)