Cover Image

MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Wenting, Zhu, Zhongrui, Huang, Guolin, Wang, Wenxuan
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models
by: Ding, Meidan, et al.
Published: (2025)

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation
by: Zhang, Ming, et al.
Published: (2025)

MedReflect: Teaching Medical LLMs to Self-Improve via Reflective Correction
by: Huang, Yue, et al.
Published: (2025)

MedMeta: A Benchmark for LLMs in Synthesizing Meta-Analysis Conclusion from Medical Studies
by: Ha, Huy Hoang, et al.
Published: (2026)

MedEthicsQA: A Comprehensive Question Answering Benchmark for Medical Ethics Evaluation of LLMs
by: Wei, Jianhui, et al.
Published: (2025)

ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs
by: Ding, Hongxin, et al.
Published: (2025)

Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence
by: Liu, Jie, et al.
Published: (2024)

MedExQA: Medical Question Answering Benchmark with Multiple Explanations
by: Kim, Yunsoo, et al.
Published: (2024)

MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Models
by: Liu, Mianxin, et al.
Published: (2024)

Medical Dialogue Generation via Intuitive-then-Analytical Differential Diagnosis
by: Xu, Kaishuai, et al.
Published: (2024)

MedFact: Benchmarking the Fact-Checking Capabilities of Large Language Models on Chinese Medical Texts
by: He, Jiayi, et al.
Published: (2025)

MedAgentsBench: Benchmarking Thinking Models and Agent Frameworks for Complex Medical Reasoning
by: Tang, Xiangru, et al.
Published: (2025)

Med-U1: Incentivizing Unified Medical Reasoning in LLMs via Large-scale Reinforcement Learning
by: Zhang, Xiaotian, et al.
Published: (2025)

MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
by: Xia, Shujun, et al.
Published: (2025)

RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning
by: Jin, Congyun, et al.
Published: (2024)

MedMT-Bench: Can LLMs Memorize and Understand Long Multi-Turn Conversations in Medical Scenarios?
by: Yang, Lin, et al.
Published: (2026)

Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models
by: Chen, Wenting, et al.
Published: (2025)

MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
by: Wu, Juncheng, et al.
Published: (2025)

MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models
by: Pandit, Shrey, et al.
Published: (2025)

MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQA
by: Wen, Shengtao, et al.
Published: (2025)

MedCalc-Eval and MedCalc-Env: Advancing Medical Calculation Capabilities of Large Language Models
by: Mao, Kangkun, et al.
Published: (2025)

LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals
by: Toker, Gilat, et al.
Published: (2026)

MedRepBench: A Comprehensive Benchmark for Medical Report Interpretation
by: Shang, Fangxin, et al.
Published: (2025)

MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
by: Zuo, Kaiwen, et al.
Published: (2024)

MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio
by: Rajgarhia, Harshit, et al.
Published: (2026)

SpeechMedAssist: Efficiently and Effectively Adapting Speech Language Models for Medical Consultation
by: Chen, Sirry, et al.
Published: (2026)

MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding
by: Zuo, Yuxin, et al.
Published: (2025)

AdaThink-Med: Medical Adaptive Thinking with Uncertainty-Guided Length Calibration
by: Rui, Shaohao, et al.
Published: (2025)

InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoning
by: Liu, Zeyu, et al.
Published: (2025)

PerMedCQA: Benchmarking Large Language Models on Medical Consumer Question Answering in Persian Language
by: Jamali, Naghmeh, et al.
Published: (2025)

Can LLMs Explain Themselves Counterfactually?
by: Dehghanighobadi, Zahra, et al.
Published: (2025)

MedArabiQ: Benchmarking Large Language Models on Arabic Medical Tasks
by: Daoud, Mouath Abu, et al.
Published: (2025)

MedAction: Towards Active Multi-turn Clinical Diagnostic LLMs
by: Hsu, Hsin-Ling, et al.
Published: (2026)

Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
by: Wang, Wenxuan, et al.
Published: (2025)

MedRECT: A Medical Reasoning Benchmark for Error Correction in Clinical Texts
by: Iwase, Naoto, et al.
Published: (2025)

DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models
by: Zhu, Yakun, et al.
Published: (2025)

CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoning
by: Onyame, Eric, et al.
Published: (2026)

LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models
by: Yang, Hang, et al.
Published: (2024)

MedFabric and EtHER: A Data-Centric Framework for Word-Level Fabrication Generation and Detection in Medical LLMs
by: Kwok, Tung Sum Thomas, et al.
Published: (2026)

SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia
by: Liu, Chaoqun, et al.
Published: (2025)