Cover Image

Modularity in Transformers: Investigating Neuron Separability & Specialization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pochinkov, Nicholas, Jones, Thomas, Rahman, Mohammed Rashidur
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language 68T07 (Primary) 68Q32, 68T05 (Secondary) I.2.4; I.2.6; I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Investigating Neuron Ablation in Attention Heads: The Case for Peak Activation Centering
by: Pochinkov, Nicholas, et al.
Published: (2024)

Think Thrice Before You Speak: Dual knowledge-enhanced Theory-of-Mind Reasoning for Persuasive Agents
by: Ma, Minghui, et al.
Published: (2026)

InhibiDistilbert: Knowledge Distillation for a ReLU and Addition-based Transformer
by: Zhang, Tony, et al.
Published: (2025)

CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density
by: Kaiser, Daniel, et al.
Published: (2025)

Rule Extraction in Machine Learning: Chat Incremental Pattern Constructor
by: Nwokocha, Caleb Princewill
Published: (2022)

Theoretical Analysis of Positional Encodings in Transformer Models: Impact on Expressiveness and Generalization
by: Li, Yin
Published: (2025)

Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology
by: Da, Longchao, et al.
Published: (2025)

Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering
by: Chen, Tiejin, et al.
Published: (2026)

Energy-Efficient Information Representation in MNIST Classification Using Biologically Inspired Learning
by: Stricker, Patrick, et al.
Published: (2026)

Do Reasoning Models Enhance Embedding Models?
by: Chan, Wun Yu, et al.
Published: (2026)

Research on a hybrid LSTM-CNN-Attention model for text-based web content classification
by: Kuz, Mykola, et al.
Published: (2025)

Linguistic Collapse: Neural Collapse in (Large) Language Models
by: Wu, Robert, et al.
Published: (2024)

The Lock-In Phase Hypothesis: Identity Consolidation as a Precursor to AGI
by: Amaral, Marcelo Maciel, et al.
Published: (2025)

NeuronSpark: A Spiking Neural Network Language Model with Selective State Space Dynamics
by: Tang, Zhengzheng
Published: (2026)

Neuro-Symbolic Process Anomaly Detection
by: Gaikwad, Devashish, et al.
Published: (2026)

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution
by: Liu, Xiaoou, et al.
Published: (2026)

Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining
by: Cao, Deyu, et al.
Published: (2025)

When Does Content-Based Routing Work? Representation Requirements for Selective Attention in Hybrid Sequence Models
by: Basu, Abhinaba
Published: (2026)

MAcPNN: Mutual Assisted Learning on Data Streams with Temporal Dependence
by: Giannini, Federico, et al.
Published: (2026)

NOTAI.AI: Explainable Detection of Machine-Generated Text via Curvature and Feature Attribution
by: Breneur, Oleksandr Marchenko, et al.
Published: (2026)

Rethinking the Multilingual Reasoning Gap with Layer Swap
by: Lasbordes, Maxence, et al.
Published: (2026)

Approaches to Semantic Textual Similarity in Slovak Language: From Algorithms to Transformers
by: Radosky, Lukas, et al.
Published: (2026)

Graded Transformers
by: Shaska Sr, Tony
Published: (2025)

On measuring grounding and generalizing grounding problems
by: Quigley, Daniel, et al.
Published: (2025)

Harnessing non-adversarial robustness in large language models
by: Zhou, Qinghua, et al.
Published: (2026)

Automated but Atrophied? Student Over-Reliance vs Expert Augmentation of AI in Learning and Cybersecurity
by: Khan, Koffka
Published: (2025)

Thinking Machines: Mathematical Reasoning in the Age of LLMs
by: Asperti, Andrea, et al.
Published: (2025)

CLMN: Concept based Language Models via Neural Symbolic Reasoning
by: Yang, Yibo
Published: (2025)

Backpropagation Through Time For Networks With Long-Term Dependencies
by: Bird, George, et al.
Published: (2021)

The Concept Allocation Zone: Tracking How Concepts Form Across Transformer Depth
by: Henry, James
Published: (2026)

A Survey on Vision-Language-Action Models for Embodied AI
by: Ma, Yueen, et al.
Published: (2024)

Latent Object Permanence: Topological Phase Transitions, Free-Energy Principles, and Renormalization Group Flows in Deep Transformer Manifolds
by: Alpay, Faruk, et al.
Published: (2026)

Sliced-Wasserstein Distribution Alignment Loss Improves the Ultra-Low-Bit Quantization of Large Language Models
by: Cao, Deyu, et al.
Published: (2026)

Causal Dimensionality of Transformer Representations: Measurement, Scaling, and Layer Structure
by: Sarkar, Nilesh, et al.
Published: (2026)

A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
by: Caunhye, Ali Murtaza, et al.
Published: (2025)

Semantic Retention and Extreme Compression in LLMs: Can We Have Both?
by: Laborde, Stanislas, et al.
Published: (2025)

Feature Selection Based on Reinforcement Learning and Hazard State Classification for Magnetic Adhesion Wall-Climbing Robots
by: Ma, Zhen, et al.
Published: (2025)

How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models
by: Borobia, Hector, et al.
Published: (2026)

AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Models
by: Keeman, Michael
Published: (2026)

A Practical Guide to Streaming Continual Learning
by: Cossu, Andrea, et al.
Published: (2026)