Cover Image

Mechanistic Insights into Grokking from the Embedding Layer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	AlquBoj, H. V., AlQuabeh, Hilal, Bojkovic, Velibor, Nwadike, Munachiso, Inui, Kentaro
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Number Representations in LLMs: A Computational Parallel to Human Perception
by: AlquBoj, H. V., et al.
Published: (2025)

Emergence of Primacy and Recency Effect in Mamba: A Mechanistic Point of View
by: Airlangga, Muhammad Cendekia, et al.
Published: (2025)

Uncovering the Spectral Bias in Diagonal State Space Models
by: Solozabal, Ruben, et al.
Published: (2025)

RECALL: Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles
by: Nwadike, Munachiso, et al.
Published: (2025)

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
by: El-Shangiti, Ahmed Oumar, et al.
Published: (2024)

Limited Memory Online Gradient Descent for Kernelized Pairwise Learning with Dynamic Averaging
by: AlQuabeh, Hilal, et al.
Published: (2024)

Sycophancy Hides Linearly in the Attention Heads
by: Genadi, Rifo, et al.
Published: (2026)

WaveSSM: Multiscale State-Space Models for Non-stationary Signal Attention
by: Solozabal, Ruben, et al.
Published: (2026)

The AI Data Scientist
by: Akimov, Farkhad, et al.
Published: (2025)

Measuring AI Reasoning: A Guide for Researchers
by: Nwadike, Munachiso Samuel, et al.
Published: (2026)

Constrained Adversarial Perturbation
by: Nishad, Virendra, et al.
Published: (2025)

Improving Personalisation in Valence and Arousal Prediction using Data Augmentation
by: Nwadike, Munachiso, et al.
Published: (2024)

Temporal Misalignment in ANN-SNN Conversion and Its Mitigation via Probabilistic Spiking Neurons
by: Bojković, Velibor, et al.
Published: (2025)

How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders
by: Inaba, Tatsuro, et al.
Published: (2025)

RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
by: Sun, Yiyou, et al.
Published: (2025)

Scaling Embedding Layers in Language Models
by: Yu, Da, et al.
Published: (2025)

Critical Data Size of Language Models from a Grokking Perspective
by: Zhu, Xuekai, et al.
Published: (2024)

Towards Quantifying Commonsense Reasoning with Mechanistic Insights
by: Joshi, Abhinav, et al.
Published: (2025)

Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes
by: Hanqing, Liu, et al.
Published: (2026)

An Analysis of Embedding Layers and Similarity Scores using Siamese Neural Networks
by: Bingi, Yash, et al.
Published: (2023)

Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization
by: Guo, Phillip, et al.
Published: (2024)

Residual Speech Embeddings for Tone Classification: Removing Linguistic Content to Enhance Paralinguistic Analysis
by: Ahbabi, Hamdan Al, et al.
Published: (2025)

Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
by: Liu, Xuyuan, et al.
Published: (2025)

Exploring Grokking: Experimental and Mechanistic Investigations
by: Qiye, Hu, et al.
Published: (2024)

On Mechanistic Circuits for Extractive Question-Answering
by: Basu, Samyadeep, et al.
Published: (2025)

Mechanistic Interpretability of Binary and Ternary Transformers
by: Li, Jason
Published: (2024)

What Makes and Breaks Safety Fine-tuning? A Mechanistic Study
by: Jain, Samyak, et al.
Published: (2024)

Spelling-out is not Straightforward: LLMs' Capability of Tokenization from Token to Characters
by: Hiraoka, Tatsuya, et al.
Published: (2025)

Mechanistic?
by: Saphra, Naomi, et al.
Published: (2024)

Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification
by: ElNokrashy, Muhammad, et al.
Published: (2022)

Mechanistic Anomaly Detection for "Quirky" Language Models
by: Johnston, David O., et al.
Published: (2025)

Triangulation as an Acceptance Rule for Multilingual Mechanistic Interpretability
by: Long, Yanan
Published: (2025)

Simple Mechanistic Explanations for Out-Of-Context Reasoning
by: Wang, Atticus, et al.
Published: (2025)

LOLAMEME: Logic, Language, Memory, Mechanistic Framework
by: Desai, Jay, et al.
Published: (2024)

Tracking Equivalent Mechanistic Interpretations Across Neural Networks
by: Sun, Alan, et al.
Published: (2026)

GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning
by: Solatorio, Aivin V.
Published: (2024)

Tracking the Feature Dynamics in LLM Training: A Mechanistic Study
by: Xu, Yang, et al.
Published: (2024)

Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classification
by: Eshuijs, Leon, et al.
Published: (2025)

Mechanistic Exploration of Backdoored Large Language Model Attention Patterns
by: Baker, Mohammed Abu, et al.
Published: (2025)

Eliminating Position Bias of Language Models: A Mechanistic Approach
by: Wang, Ziqi, et al.
Published: (2024)