Cover Image

LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Lingyao, Li, Deyi, Chen, Chen, Ma, Renkai, Yu, Runlong, Lin, Mingquan, Yin, Rui, Fan, Lizhou, Shyr, Cathy, Ma, Siyuan, Liu, Mei, Bethard, Steven
Format:	Preprint
Published:	2026
Subjects:	Computers and Society
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers
by: Li, Lingyao, et al.
Published: (2026)

LSDTs: LLM-Augmented Semantic Digital Twins for Adaptive Knowledge-Intensive Infrastructure Planning
by: Li, Naiyi, et al.
Published: (2025)

The Rise of AI Agent Communities: Large-Scale Analysis of Discourse and Interaction on Moltbook
by: Li, Lingyao, et al.
Published: (2026)

Beyond the Uncanny Valley: A Mixed-Method Investigation of Anthropomorphism in Protective Responses to Robot Abuse
by: Yang, Fan, et al.
Published: (2025)

Whether We Care, How We Reason: The Dual Role of Anthropomorphism and Moral Foundations in Robot Abuse
by: Yang, Fan, et al.
Published: (2026)

Towards Trustworthy AI: Characterizing User-Reported Risks across LLMs "In the Wild"
by: Li, Lingyao, et al.
Published: (2025)

LLM Use for Mental Health: Crowdsourcing Users' Sentiment-based Perspectives and Values from Social Discussions
by: Li, Lingyao, et al.
Published: (2025)

When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Review
by: Zhu, Changjia, et al.
Published: (2025)

"I Use ChatGPT to Humanize My Words": Affordances and Risks of ChatGPT to Autistic Users
by: Ma, Renkai, et al.
Published: (2026)

I don't Want You to Die: A Shared Responsibility Framework for Safeguarding Child-Robot Companionship
by: Yang, Fan, et al.
Published: (2025)

Empowering LLM Agents with Geospatial Awareness: Toward Grounded Reasoning for Wildfire Response
by: Chen, Yiheng, et al.
Published: (2025)

Characterizing Online Toxicity During the 2022 Mpox Outbreak: A Computational Analysis of Topical and Network Dynamics
by: Fan, Lizhou, et al.
Published: (2024)

Transformer-Based Temporal Information Extraction and Application: A Review
by: Su, Xin, et al.
Published: (2025)

"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media
by: Li, Lingyao, et al.
Published: (2023)

Negotiating Digital Identities with AI Companions: Motivations, Strategies, and Emotional Outcomes
by: Ma, Renkai, et al.
Published: (2026)

LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment
by: Li, Lingyao, et al.
Published: (2025)

A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework
by: Li, Chenyu, et al.
Published: (2026)

When Earth Foundation Models Meet Diffusion: An Application to Land Surface Temperature Super-Resolution
by: Chen, Yiheng, et al.
Published: (2026)

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
by: Lai, Peng, et al.
Published: (2025)

Multi-Study R-Learner for Estimating Heterogeneous Treatment Effects Across Studies Using Statistical Machine Learning
by: Shyr, Cathy, et al.
Published: (2023)

Politicized Attention Shifts Amplify Polarization in the Information Ecosystem during California Wildfires
by: Chen, Yiheng, et al.
Published: (2026)

Propensity Score‐Matched Comparison of Robotic and Laparoscopic Pancreaticoduodenectomy
by: Bor‐Shiuan Shyr, et al.
Published: (2025)

Impact of Obesity on Perioperative Outcomes in Robotic Pancreaticoduodenectomy: A Propensity Score‐Matched Study
by: Bor‐Uei Shyr, et al.
Published: (2024)

Identifying Task Groupings for Multi-Task Learning Using Pointwise V-Usable Information
by: Li, Yingya, et al.
Published: (2024)

Deciphering Scientific Collaboration in Biomedical LLM Research: Dynamics, Institutional Participation, and Resource Disparities
by: Li, Lingyao, et al.
Published: (2025)

Identifying and Extracting Rare Disease Phenotypes with Large Language Models
by: Shyr, Cathy, et al.
Published: (2023)

NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
by: Fan, Lizhou, et al.
Published: (2023)

DT-BEHRT: Disease Trajectory-aware Transformer for Interpretable Patient Representation Learning
by: Li, Deyi, et al.
Published: (2026)

Exploring Needs and Design Opportunities for Proactive Information Support in In-Person Small-Group Conversations
by: Zhou, Shaoze, et al.
Published: (2026)

Best Arm Identification with LLM Judges and Limited Human
by: Ao, Ruicheng, et al.
Published: (2026)

Toward satisfactory public accessibility: A crowdsourcing approach through online reviews to inclusive urban design
by: Li, Lingyao, et al.
Published: (2024)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
by: Shi, Lin, et al.
Published: (2024)

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
by: Lai, Peng, et al.
Published: (2026)

War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars
by: Hua, Wenyue, et al.
Published: (2023)

A Semantic Parsing Framework for End-to-End Time Normalization
by: Su, Xin, et al.
Published: (2025)

Semi-Structured Chain-of-Thought: Integrating Multiple Sources of Knowledge for Improved Language Model Reasoning
by: Su, Xin, et al.
Published: (2023)

Interactions Between Carbon Fibre and Thermoplastics
by: T. Shyr, et al.
Published: (1995)

Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale
by: Li, Weiyue, et al.
Published: (2026)

RAG for Geoscience: What We Expect, Gaps and Opportunities
by: Yu, Runlong, et al.
Published: (2025)

Recent advances in deep learning and language models for studying the microbiome
by: Yan, Binghao, et al.
Published: (2024)