Cover Image

AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Xiechi, Ouyang, Zetian, Wang, Linlin, de Melo, Gerard, Cao, Zhu, Wang, Xiaoling, Zhang, Ya, Wang, Yanfeng, He, Liang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ACE-$M^3$: Automatic Capability Evaluator for Multimodal Medical Models
by: Zhang, Xiechi, et al.
Published: (2024)

CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios
by: Ouyang, Zetian, et al.
Published: (2024)

Hierarchical Divide-and-Conquer for Fine-Grained Alignment in LLM-Based Medical Evaluation
by: Zheng, Shunfan, et al.
Published: (2025)

NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning
by: Yi, Xin, et al.
Published: (2024)

DictLLM: Harnessing Key-Value Data Structures with Large Language Models for Enhanced Medical Diagnostics
by: Guo, YiQiu, et al.
Published: (2024)

MedCalc-Eval and MedCalc-Env: Advancing Medical Calculation Capabilities of Large Language Models
by: Mao, Kangkun, et al.
Published: (2025)

MedKAN: An Advanced Kolmogorov-Arnold Network for Medical Image Classification
by: Yang, Zhuoqin, et al.
Published: (2025)

MedS$^3$: Towards Medical Slow Thinking with Self-Evolved Soft Dual-sided Process Supervision
by: Jiang, Shuyang, et al.
Published: (2025)

Fine-Grained Detoxification via Instance-Level Prefixes for Large Language Models
by: Yi, Xin, et al.
Published: (2024)

MedEthicEval: Evaluating Large Language Models Based on Chinese Medical Ethics
by: Jin, Haoan, et al.
Published: (2025)

Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models
by: Li, Yue, et al.
Published: (2025)

PhenoLIP: Integrating Phenotype Ontology Knowledge into Medical Vision-Language Pretraining
by: Liang, Cheng, et al.
Published: (2026)

Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images
by: Huang, Chaoqin, et al.
Published: (2024)

MultiMedEval: A Benchmark and a Toolkit for Evaluating Medical Vision-Language Models
by: Royer, Corentin, et al.
Published: (2024)

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models
by: Liu, Junqi, et al.
Published: (2026)

Benchmarking Egocentric Clinical Intent Understanding Capability for Medical Multimodal Large Language Models
by: Liu, Shaonan, et al.
Published: (2026)

Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Education
by: Yi, Xin, et al.
Published: (2025)

MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
by: Liao, Yusheng, et al.
Published: (2024)

Language-guided Scale-aware MedSegmentor for Lesion Segmentation in Medical Imaging
by: Ouyang, Shuyi, et al.
Published: (2024)

EffiEval: Efficient and Generalizable Model Evaluation via Capability Coverage Maximization
by: Wang, Yaoning, et al.
Published: (2025)

ReEvalMed: Rethinking Medical Report Evaluation by Aligning Metrics with Real-World Clinical Judgment
by: Li, Ruochen, et al.
Published: (2025)

A safety realignment framework via subspace-oriented model fusion for large language models
by: Yi, Xin, et al.
Published: (2024)

Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator
by: Liao, Yusheng, et al.
Published: (2024)

MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning
by: Pan, Jiazhen, et al.
Published: (2025)

MedRiskEval: Medical Risk Evaluation Benchmark of Language Models, On the Importance of User Perspectives in Healthcare Settings
by: Corbeil, Jean-Philippe, et al.
Published: (2025)

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models
by: Xu, Dexuan, et al.
Published: (2025)

StreamingEval: A Unified Evaluation Protocol towards Realistic Streaming Video Understanding
by: Tang, Guowei, et al.
Published: (2026)

MedFact: Benchmarking the Fact-Checking Capabilities of Large Language Models on Chinese Medical Texts
by: He, Jiayi, et al.
Published: (2025)

CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluation
by: Zhao, Jingqian, et al.
Published: (2025)

T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step
by: Chen, Zehui, et al.
Published: (2023)

ProtoMed-LLM: An Automatic Evaluation Framework for Large Language Models in Medical Protocol Formulation
by: Yi, Seungjun, et al.
Published: (2024)

AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering
by: Chen, Xiuyuan, et al.
Published: (2023)

Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models
by: Ding, Meidan, et al.
Published: (2025)

Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm
by: Liu, Hongcheng, et al.
Published: (2024)

MedSAM-U: Uncertainty-Guided Auto Multi-Prompt Adaptation for Reliable MedSAM
by: Zhou, Nan, et al.
Published: (2024)

ScalerEval: Automated and Consistent Evaluation Testbed for Auto-scalers in Microservices
by: Xie, Shuaiyu, et al.
Published: (2025)

MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Models
by: Liu, Mianxin, et al.
Published: (2024)

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models
by: Kwan, Wai-Chung, et al.
Published: (2024)

VulnRepairEval: An Exploit-Based Evaluation Framework for Assessing Large Language Model Vulnerability Repair Capabilities
by: Wang, Weizhe, et al.
Published: (2025)

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
by: Lin, Minhua, et al.
Published: (2026)