Cover Image

Attention Mechanisms Don't Learn Additive Models: Rethinking Feature Importance for Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Leemann, Tobias, Fastowski, Alina, Pfeiffer, Felix, Kasneci, Gjergji
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Understanding Knowledge Drift in LLMs through Misinformation
by: Fastowski, Alina, et al.
Published: (2024)

From Confidence to Collapse in LLM Factual Robustness
by: Fastowski, Alina, et al.
Published: (2025)

Injecting Falsehoods: Adversarial Man-in-the-Middle Attacks Undermining Factual Recall in LLMs
by: Fastowski, Alina, et al.
Published: (2025)

Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents
by: Kirchhof, Michael, et al.
Published: (2025)

I Prefer not to Say: Protecting User Consent in Models with Optional Personal Data
by: Leemann, Tobias, et al.
Published: (2022)

Emergent Abilities in Large Language Models: A Survey
by: Berti, Leonardo, et al.
Published: (2025)

CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models
by: Kocak, Aysenur, et al.
Published: (2025)

EvalxNLP: A Framework for Benchmarking Post-Hoc Explainability Methods on NLP Models
by: Dhaini, Mahdi, et al.
Published: (2025)

Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods
by: Dhaini, Mahdi, et al.
Published: (2025)

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
by: Abramov, Roman, et al.
Published: (2025)

Don't Shoot The Breeze: Topic Continuity Model Using Nonlinear Naive Bayes With Attention
by: Pi, Shu-Ting, et al.
Published: (2026)

Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models
by: Zhang, Zheyu, et al.
Published: (2025)

Enriching Tabular Data with Contextual LLM Embeddings: A Comprehensive Ablation Study for Ensemble Classifiers
by: Kasneci, Gjergji, et al.
Published: (2024)

TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data
by: Berti, Leonardo, et al.
Published: (2025)

Reasoning Models Don't Always Say What They Think
by: Chen, Yanda, et al.
Published: (2025)

Is Crowdsourcing Breaking Your Bank? Cost-Effective Fine-Tuning of Pre-trained Language Models with Proximal Policy Optimization
by: Yang, Shuo, et al.
Published: (2024)

Large Language Models Must Be Taught to Know What They Don't Know
by: Kapoor, Sanyam, et al.
Published: (2024)

You Don't Need Prompt Engineering Anymore: The Prompting Inversion
by: Khan, Imran
Published: (2025)

Don't Trust: Verify -- Grounding LLM Quantitative Reasoning with Autoformalization
by: Zhou, Jin Peng, et al.
Published: (2024)

Why Don't Prompt-Based Fairness Metrics Correlate?
by: Zayed, Abdelrahman, et al.
Published: (2024)

Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't
by: Svete, Anej, et al.
Published: (2026)

LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations
by: Mayne, Harry, et al.
Published: (2025)

Don't Pay Attention
by: Hammoud, Mohammad, et al.
Published: (2025)

Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models
by: Yuan, Chenchen, et al.
Published: (2026)

Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?
by: Jedidi, Nour, et al.
Published: (2025)

Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation
by: Hariri, Mohsen, et al.
Published: (2025)

The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't
by: Shin, Kwan Soo
Published: (2026)

LLM Cyber Evaluations Don't Capture Real-World Risk
by: Lukošiūtė, Kamilė, et al.
Published: (2025)

Adoption of Explainable Natural Language Processing: Perspectives from Industry and Academia on Practices and Challenges
by: Dhaini, Mahdi, et al.
Published: (2025)

Analysing the Safety Pitfalls of Steering Vectors
by: Li, Yuxiao, et al.
Published: (2026)

Towards Non-Adversarial Algorithmic Recourse
by: Leemann, Tobias, et al.
Published: (2024)

Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions
by: Parmar, Mihir, et al.
Published: (2022)

Multimodal Assessment of Classroom Discourse Quality: A Text-Centered Attention-Based Multi-Task Learning Approach
by: Hou, Ruikun, et al.
Published: (2025)

Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Models
by: Zausinger, Jonas, et al.
Published: (2024)

Don't Forget to Connect! Improving RAG with Graph-based Reranking
by: Dong, Jialin, et al.
Published: (2024)

Comparing Feature Importance and Rule Extraction for Interpretability on Text Data
by: Lopardo, Gianluigi, et al.
Published: (2022)

Consolidating Rewarded Perturbations for LLM Post-Training
by: Zhang, Zheyu, et al.
Published: (2026)

RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting
by: Yang, Shuo, et al.
Published: (2024)

Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
by: Liu, Jiacheng, et al.
Published: (2023)

Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
by: Choi, Minsik, et al.
Published: (2025)