Cover Image

Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ren, Zhiyao, Zhan, Yibing, Liang, Siyuan, Ma, Guozheng, Yu, Baosheng, Tao, Dacheng
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation
by: Ren, Zhiyao, et al.
Published: (2024)

Reverse Prompt: Cracking the Recipe Inside Text-to-Image Generation
by: Ren, Zhiyao, et al.
Published: (2025)

NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models
by: Ho, Zheng Yi, et al.
Published: (2024)

Re-Initialization Token Learning for Tool-Augmented Large Language Models
by: Li, Chenghao, et al.
Published: (2025)

EMRModel: A Large Language Model for Extracting Medical Consultation Dialogues into Structured Medical Records
by: Zhao, Shuguang, et al.
Published: (2025)

Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning
by: Zhang, Yanfang, et al.
Published: (2024)

Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models
by: Huang, Zhongzhen, et al.
Published: (2024)

Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
by: Zan, Changtong, et al.
Published: (2024)

MedDialogRubrics: A Comprehensive Benchmark and Evaluation Framework for Multi-turn Medical Consultations in Large Language Models
by: Gong, Lecheng, et al.
Published: (2026)

Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
by: Xiao, Yisong, et al.
Published: (2025)

Uncertainty Estimation of Large Language Models in Medical Question Answering
by: Wu, Jiaxin, et al.
Published: (2024)

OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
by: Wang, Shuai, et al.
Published: (2024)

Large Language Model Benchmarks in Medical Tasks
by: Yan, Lawrence K. Q., et al.
Published: (2024)

AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
by: Fan, Zhihao, et al.
Published: (2024)

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
by: Xia, Tianle, et al.
Published: (2024)

How Reliable are Confidence Estimators for Large Reasoning Models? A Systematic Benchmark on High-Stakes Domains
by: Khanmohammadi, Reza, et al.
Published: (2026)

Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation
by: Lim, Seungseop, et al.
Published: (2025)

Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators
by: Mahaut, Matéo, et al.
Published: (2024)

League of LLMs: A Benchmark-Free Paradigm for Mutual Evaluation of Large Language Models
by: Guo, Qianhong, et al.
Published: (2025)

A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Models
by: Zhang, Junjie, et al.
Published: (2025)

MedicalBench: Evaluating Large Language Models Toward Improved Medical Concept Extraction
by: Yang, Zhichao, et al.
Published: (2026)

Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models
by: Chen, Wenting, et al.
Published: (2025)

MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Models
by: Liu, Mianxin, et al.
Published: (2024)

Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence
by: Lu, Yuyin, et al.
Published: (2026)

Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations
by: Li, Yahan, et al.
Published: (2026)

Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Models
by: Pham, Duy Khoa, et al.
Published: (2024)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
by: Li, Hongyu, et al.
Published: (2024)

MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMs
by: Qu, Zhan, et al.
Published: (2025)

Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions
by: Chen, Hanjie, et al.
Published: (2024)

Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
by: Vishwanath, Krithik, et al.
Published: (2025)

Medical Large Language Model Benchmarks Should Prioritize Construct Validity
by: Alaa, Ahmed, et al.
Published: (2025)

ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks
by: Ren, Zhiyao, et al.
Published: (2025)

IPEval: A Bilingual Intellectual Property Agency Consultation Evaluation Benchmark for Large Language Models
by: Wang, Qiyao, et al.
Published: (2024)

Towards Training A Chinese Large Language Model for Anesthesiology
by: Wang, Zhonghai, et al.
Published: (2024)

Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study
by: Zhan, Zaifu, et al.
Published: (2026)

SpeechMedAssist: Efficiently and Effectively Adapting Speech Language Models for Medical Consultation
by: Chen, Sirry, et al.
Published: (2026)

Multi-Perspective Consistency Enhances Confidence Estimation in Large Language Models
by: Wang, Pei, et al.
Published: (2024)

Medical Reasoning with Large Language Models: A Survey and MR-Bench
by: Ren, Xiaohan, et al.
Published: (2026)

Comparing Large Language Models and Traditional Machine Translation Tools for Translating Medical Consultation Summaries: A Pilot Study
by: Li, Andy, et al.
Published: (2025)

PeruMedQA: Benchmarking Large Language Models (LLMs) on Peruvian Medical Exams -- Dataset Construction and Evaluation
by: Carrillo-Larco, Rodrigo M., et al.
Published: (2025)