Image de couverture de livre

Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Turk, Matt
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

AgentBench: Evaluating LLMs as Agents
par: Liu, Xiao, et autres
Publié: (2023)

STEM: Efficient Relative Capability Evaluation of LLMs through Structured Transition Samples
par: Hu, Haiquan, et autres
Publié: (2025)

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
par: Zhang, Hanlin, et autres
Publié: (2026)

Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning
par: Zhou, Qinhao, et autres
Publié: (2024)

PersonaGym: Evaluating Persona Agents and LLMs
par: Samuel, Vinay, et autres
Publié: (2024)

ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
par: Oh, Jungwoo, et autres
Publié: (2026)

CTBench: A Comprehensive Benchmark for Evaluating Language Model Capabilities in Clinical Trial Design
par: Neehal, Nafis, et autres
Publié: (2024)

Lita: Light Agent Uncovers the Agentic Coding Capabilities of LLMs
par: Dai, Hankun, et autres
Publié: (2025)

Quantifying the Capabilities of LLMs across Scale and Precision
par: Badshah, Sher, et autres
Publié: (2024)

Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
par: Hamman, Faisal, et autres
Publié: (2025)

Exploring the Hidden Capacity of LLMs for One-Step Text Generation
par: Mezentsev, Gleb, et autres
Publié: (2025)

Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find
par: Bianchi, Owen, et autres
Publié: (2025)

IITK at SemEval-2024 Task 2: Exploring the Capabilities of LLMs for Safe Biomedical Natural Language Inference for Clinical Trials
par: Mandal, Shreyasi, et autres
Publié: (2024)

PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities
par: Li, Haoming, et autres
Publié: (2025)

LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations
par: Mayne, Harry, et autres
Publié: (2025)

Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Exploration
par: Deng, Wenhao, et autres
Publié: (2025)

How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs
par: Feng, Guhao, et autres
Publié: (2024)

From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
par: Schiekiera, Louis, et autres
Publié: (2026)

Direction-Flipped Influence Audits Reveal Hidden Structure in Moral Choices of LLMs
par: Blandfort, Phil, et autres
Publié: (2026)

Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
par: Baumann, Joachim, et autres
Publié: (2025)

When Reasoning Hurts: Source-Aware Evaluation of Frontier LLMs for Clinical SOAP Note Generation
par: Faisal, Faizan
Publié: (2026)

Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
par: Dua, Radhika, et autres
Publié: (2025)

EffGen: Enabling Small Language Models as Capable Autonomous Agents
par: Srivastava, Gaurav, et autres
Publié: (2026)

CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine
par: Guo, Kevin H., et autres
Publié: (2026)

DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life
par: Chiu, Yu Ying, et autres
Publié: (2024)

Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Tasks
par: Soni, Nikita, et autres
Publié: (2025)

Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
par: Lin, Xiaofeng, et autres
Publié: (2026)

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
par: DeepSeek-AI, et autres
Publié: (2025)

Can We Count on LLMs? The Fixed-Effect Fallacy and Claims of GPT-4 Capabilities
par: Ball, Thomas, et autres
Publié: (2024)

Zero-shot LLM-guided Counterfactual Generation: A Case Study on NLP Model Evaluation
par: Bhattacharjee, Amrita, et autres
Publié: (2024)

Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks
par: Hegazy, Mahmood
Publié: (2024)

CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities
par: Mao, Yujun, et autres
Publié: (2024)

RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization
par: Dong, Yihong, et autres
Publié: (2025)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
par: Tian, Yijun, et autres
Publié: (2024)

Expanding Foundational Language Capabilities in Open-Source LLMs through a Korean Case Study
par: Lim, Junghwan, et autres
Publié: (2025)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
par: Xiong, Zheyang, et autres
Publié: (2024)

Evaluation is All You Need: Strategic Overclaiming of LLM Reasoning Capabilities Through Evaluation Design
par: Sun, Lin, et autres
Publié: (2025)

Are LLM Agents Behaviorally Coherent? Latent Profiles for Social Simulation
par: Mooney, James, et autres
Publié: (2025)

CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
par: Feng, Jie, et autres
Publié: (2024)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
par: Liu, Yixin, et autres
Publié: (2025)