Image de couverture de livre

VERT: Reliable LLM Judges for Radiology Report Evaluation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bologna, Federica, Corbeil, Jean-Philippe, Wilkens, Matthew, Abacha, Asma Ben
Format:	Preprint
Publié:	2026
Sujets:	Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

CQA-Eval: Designing Reliable Evaluations of Multi-paragraph Clinical QA under Resource Constraints
par: Bologna, Federica, et autres
Publié: (2025)

Overview of the MEDIQA-OE 2025 Shared Task on Medical Order Extraction from Doctor-Patient Consultations
par: Corbeil, Jean-Philippe, et autres
Publié: (2025)

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
par: Corbeil, Jean-Philippe, et autres
Publié: (2025)

IryoNLP at MEDIQA-CORR 2024: Tackling the Medical Error Detection & Correction Task On the Shoulders of Medical Agents
par: Corbeil, Jean-Philippe
Publié: (2024)

Empowering Healthcare Practitioners with Language Models: Structuring Speech Transcripts in Two Real-World Clinical Applications
par: Corbeil, Jean-Philippe, et autres
Publié: (2025)

GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents
par: Moll, Johannes, et autres
Publié: (2026)

MRScore: Evaluating Radiology Report Generation with LLM-based Reward System
par: Liu, Yunyi, et autres
Publié: (2024)

GREEN: Generative Radiology Report Evaluation and Error Notation
par: Ostmeier, Sophie, et autres
Publié: (2024)

From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges
par: Hong, Yihan, et autres
Publié: (2026)

Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reports
par: Sun, Chengbo, et autres
Publié: (2025)

Evaluating Metrics for Safety with LLM-as-Judges
par: Clegg, Kester, et autres
Publié: (2025)

MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes
par: Abacha, Asma Ben, et autres
Publié: (2024)

PARROT: An Open Multilingual Radiology Reports Dataset
par: Guellec, Bastien Le, et autres
Publié: (2025)

LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generation
par: Wang, Zilong, et autres
Publié: (2024)

A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness
par: Schwinn, Leo, et autres
Publié: (2026)

RadReason: Radiology Report Evaluation Metric with Reasons and Sub-Scores
par: Li, Yingshu, et autres
Publié: (2025)

Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports
par: Zhu, Qingqing, et autres
Publié: (2024)

DermaVQA-DAS: Dermatology Assessment Schema (DAS) & Datasets for Closed-Ended Question Answering & Segmentation in Patient-Generated Dermatology Images
par: Yim, Wen-wai, et autres
Publié: (2025)

RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
par: Shetty, Saisha Pradeep, et autres
Publié: (2026)

Process Reward Models for Sentence-Level Verification of LVLM Radiology Reports
par: Thomas, Alois, et autres
Publié: (2025)

CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluation
par: Baharoon, Mohammed, et autres
Publié: (2026)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
par: Tan, Sijun, et autres
Publié: (2024)

Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation
par: Wang, Xinyi, et autres
Publié: (2026)

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
par: Thakur, Aman Singh, et autres
Publié: (2024)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
par: Liu, Yixin, et autres
Publié: (2025)

Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
par: Saha, Swarnadeep, et autres
Publié: (2025)

Summarizing Radiology Reports Findings into Impressions
par: de Padua, Raul Salles, et autres
Publié: (2024)

Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations
par: Gupta, Manan, et autres
Publié: (2026)

CLEAR: A Clinically-Grounded Tabular Framework for Radiology Report Evaluation
par: Jiang, Yuyang, et autres
Publié: (2025)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
par: Shi, Lin, et autres
Publié: (2024)

LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluation
par: Enguehard, Joseph, et autres
Publié: (2025)

Two-Pronged Human Evaluation of ChatGPT Self-Correction in Radiology Report Simplification
par: Yang, Ziyu, et autres
Publié: (2024)

Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
par: Zhang, Xingjian, et autres
Publié: (2025)

ReFINE: A Reward-Based Framework for Interpretable and Nuanced Evaluation of Radiology Report Generation
par: Liu, Yunyi, et autres
Publié: (2024)

Do Repetitions Matter? Strengthening Reliability in LLM Evaluations
par: Gonzalez, Miguel Angel Alvarado, et autres
Publié: (2025)

JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
par: Jin, Jiho, et autres
Publié: (2026)

Case-Aware LLM-as-a-Judge Evaluation for Enterprise-Scale RAG Systems
par: Chhabra, Mukul, et autres
Publié: (2026)

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
par: Verga, Pat, et autres
Publié: (2024)

JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
par: Shi, Zhichao, et autres
Publié: (2025)

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
par: Wang, Yidong, et autres
Publié: (2025)