Cover Image

The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Research

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bai, Xiaoyan, Baumgartner, Alexander, Sun, Haojia, Holtzman, Ari, Tan, Chenhao
Format:	Preprint
Published:	2026
Subjects:	Computers and Society Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Know Thyself? On the Incapability and Implications of AI Self-Recognition
by: Bai, Xiaoyan, et al.
Published: (2025)

Moral Mazes in the Era of LLMs
by: Nguyen, Dang, et al.
Published: (2026)

On the Effectiveness and Generalization of Race Representations for Debiasing High-Stakes Decisions
by: Nguyen, Dang, et al.
Published: (2025)

HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation
by: Liu, Haokun, et al.
Published: (2025)

Literature Meets Data: A Synergistic Approach to Hypothesis Generation
by: Liu, Haokun, et al.
Published: (2024)

Hypothesis Generation with Large Language Models
by: Zhou, Yangqiaoyu, et al.
Published: (2024)

CaseSumm: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court Opinions
by: Heddaya, Mourad, et al.
Published: (2024)

LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
by: Yang, Chenghao, et al.
Published: (2025)

From Adversarial Poetry to Adversarial Tales: An Interpretability Research Agenda
by: Bisconti, Piercosma, et al.
Published: (2025)

The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
by: Si, Chenglei, et al.
Published: (2025)

The Reasoning Trap -- Logical Reasoning as a Mechanistic Pathway to Situational Awareness
by: Sahoo, Subramanyam, et al.
Published: (2026)

Forking Paths in Neural Text Generation
by: Bigelow, Eric, et al.
Published: (2024)

Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations
by: Bhagat, Kirti, et al.
Published: (2024)

Towards Execution-Grounded Automated AI Research
by: Si, Chenglei, et al.
Published: (2026)

Causal Reasoning and Large Language Models: Opening a New Frontier for Causality
by: Kıcıman, Emre, et al.
Published: (2023)

COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act
by: Guldimann, Philipp, et al.
Published: (2024)

DualAlign: Generating Clinically Grounded Synthetic Data
by: Li, Rumeng, et al.
Published: (2025)

Interpretable Recognition of Cognitive Distortions in Natural Language Texts
by: Kolonin, Anton, et al.
Published: (2025)

Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii
by: Ayonrinde, Kola, et al.
Published: (2025)

HypoEval: Hypothesis-Guided Evaluation for Natural Language Generation
by: Li, Mingxuan, et al.
Published: (2025)

A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i
by: Ayonrinde, Kola, et al.
Published: (2025)

Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy
by: Duffy, Alexander, et al.
Published: (2025)

HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks
by: Sun, Jiuding, et al.
Published: (2025)

Health Insurance Coverage Rule Interpretation Corpus: Law, Policy, and Medical Guidance for Health Insurance Coverage Understanding
by: Gartner, Mike
Published: (2025)

Beyond Accuracy: Introducing a Symbolic-Mechanistic Approach to Interpretable Evaluation
by: Habibi, Reza, et al.
Published: (2026)

CycleResearcher: Improving Automated Research via Automated Review
by: Weng, Yixuan, et al.
Published: (2024)

MIB: A Mechanistic Interpretability Benchmark
by: Mueller, Aaron, et al.
Published: (2025)

The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval
by: Tong, Zekai, et al.
Published: (2026)

Towards Autonomous Mathematics Research
by: Feng, Tony, et al.
Published: (2026)

Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Models
by: Simbeck, Katharina, et al.
Published: (2025)

IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems
by: Wu, Yuhao, et al.
Published: (2024)

CLEAR: A Clinically-Grounded Tabular Framework for Radiology Report Evaluation
by: Jiang, Yuyang, et al.
Published: (2025)

Evaluating the Performance of ChatGPT for Spam Email Detection
by: Si, Shijing, et al.
Published: (2024)

AI Sandbagging: Language Models can Strategically Underperform on Evaluations
by: van der Weij, Teun, et al.
Published: (2024)

Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey
by: Dong, Zhichen, et al.
Published: (2024)

Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models
by: Mersha, Melkamu Abay, et al.
Published: (2025)

Towards Robust Evaluation of Unlearning in LLMs via Data Transformations
by: Joshi, Abhinav, et al.
Published: (2024)

Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
by: Dua, Radhika, et al.
Published: (2025)

A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
by: Arghal, Raghu, et al.
Published: (2026)

Evaluating Prompt Engineering Techniques for Accuracy and Confidence Elicitation in Medical LLMs
by: Naderi, Nariman, et al.
Published: (2025)