Cover Image

Interpreting Transformers Through Attention Head Intervention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kadem, Mason, Zheng, Rong
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Human-Centered Ambient and Wearable Sensing for Automated Monitoring in Dementia Care: A Scoping Review
by: Kadem, Mason, et al.
Published: (2026)

Debiasing CLIP: Interpreting and Correcting Bias in Attention Heads
by: Yeo, Wei Jie, et al.
Published: (2025)

Surgical Repair of Collapsed Attention Heads in ALiBi Transformers
by: Schallon, Palmer
Published: (2026)

RecurFormer: Not All Transformer Heads Need Self-Attention
by: Yan, Ruiqing, et al.
Published: (2024)

Improving Transformers with Dynamically Composable Multi-Head Attention
by: Xiao, Da, et al.
Published: (2024)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
by: Csordás, Róbert, et al.
Published: (2023)

Knocking-Heads Attention
by: Zhou, Zhanchao, et al.
Published: (2025)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
by: Musat, Tiberiu
Published: (2024)

Head Pursuit: Probing Attention Specialization in Multimodal Transformers
by: Basile, Lorenzo, et al.
Published: (2025)

RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
by: Tang, Hanlin, et al.
Published: (2024)

Attention Heads of Large Language Models: A Survey
by: Zheng, Zifan, et al.
Published: (2024)

Bias A-head? Analyzing Bias in Transformer-Based Language Model Attention Heads
by: Yang, Yi, et al.
Published: (2023)

The Anxiety of Influence: Bloom Filters in Transformer Attention Heads
by: Balogh, Peter
Published: (2026)

Perception Is All You Need: A Neuroscience Framework for Low Cost Sensorless Gaze in HRI
by: Kadem, Mason
Published: (2026)

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention
by: Shao, Jintian, et al.
Published: (2025)

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
by: Chen, Yilong, et al.
Published: (2024)

SignAttention: On the Interpretability of Transformer Models for Sign Language Translation
by: Bianco, Pedro Alejandro Dal, et al.
Published: (2024)

S2-Attention: Hardware-Aware Context Sharding Among Attention Heads
by: Lin, Xihui, et al.
Published: (2024)

Inferring Functionality of Attention Heads from their Parameters
by: Elhelo, Amit, et al.
Published: (2024)

Head-wise Shareable Attention for Large Language Models
by: Cao, Zouying, et al.
Published: (2024)

AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection
by: Hua, Kai, et al.
Published: (2025)

Sycophancy Hides Linearly in the Attention Heads
by: Genadi, Rifo, et al.
Published: (2026)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
by: Lu, Yi, et al.
Published: (2024)

Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysis
by: Mofael, Abdullah Al, et al.
Published: (2026)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
by: Neo, Clement, et al.
Published: (2024)

NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models
by: Ho, Zheng Yi, et al.
Published: (2024)

MuDAF: Long-Context Multi-Document Attention Focusing through Contrastive Learning on Attention Heads
by: Liu, Weihao, et al.
Published: (2025)

Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
by: Ji, Tao, et al.
Published: (2025)

When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality
by: Madureira, Brielen, et al.
Published: (2024)

Uncertainty-Aware Attention Heads: Efficient Unsupervised Uncertainty Quantification for LLMs
by: Vazhentsev, Artem, et al.
Published: (2025)

From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models
by: Ma, Youmi, et al.
Published: (2026)

Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization
by: Zhang, Weixu, et al.
Published: (2026)

Adaptive Two Sided Laplace Transforms: A Learnable, Interpretable, and Scalable Replacement for Self-Attention
by: Kiruluta, Andrew
Published: (2025)

Debiasing LLMs by Masking Unfairness-Driving Attention Heads
by: Han, Tingxu, et al.
Published: (2025)

Analyzing Multi-Head Attention on Trojan BERT Models
by: Wang, Jingwei
Published: (2024)

CHAI: Clustered Head Attention for Efficient LLM Inference
by: Agarwal, Saurabh, et al.
Published: (2024)

Latent Multi-Head Attention for Small Language Models
by: Mehta, Sushant, et al.
Published: (2025)

CausalDetox: Causal Head Selection and Intervention for Language Model Detoxification
by: Wang, Yian, et al.
Published: (2026)

A Transformer with Stack Attention
by: Li, Jiaoda, et al.
Published: (2024)

Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernel
by: Yamagiwa, Hiroaki, et al.
Published: (2026)