Cover Image

Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judges

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Yuqi, Feng, Kehua, Wang, Yunfeng, Chen, Zhiwen, Lv, Chengfei, Yu, Gang, Zhang, Qiang, Ding, Keyan, Chen, Huajun
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
by: Feng, Kehua, et al.
Published: (2024)

ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning
by: Tang, Yuqi, et al.
Published: (2025)

CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
by: Feng, Kehua, et al.
Published: (2025)

SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models
by: Yu, Jing, et al.
Published: (2025)

SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
by: Feng, Kehua, et al.
Published: (2025)

Boosting LLM's Molecular Structure Elucidation with Knowledge Enhanced Tree Search Reasoning
by: Zhuang, Xiang, et al.
Published: (2025)

ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding
by: Rao, Mingyang, et al.
Published: (2026)

Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition
by: Feng, Kehua, et al.
Published: (2024)

SciToolAgent: A Knowledge Graph-Driven Scientific Agent for Multi-Tool Integration
by: Ding, Keyan, et al.
Published: (2025)

Evaluating Reward Model Generalization via Pairwise Maximum Discrepancy Competitions
by: Luo, Shunyang, et al.
Published: (2026)

RankJudge: A Multi-Turn LLM-as-a-Judge Synthetic Benchmark Generator
by: Tang, Zhenwei, et al.
Published: (2026)

Advancing biomolecular understanding and design following human instructions
by: Zhuang, Xiang, et al.
Published: (2024)

From Myopic Selection to Long-Horizon Awareness: Sequential LLM Routing for Multi-Turn Dialogue
by: Zhang, Jiarui, et al.
Published: (2026)

CyclicJudge: Mitigating Judge Bias Efficiently in LLM-based Evaluation
by: Zhu, Ziyi, et al.
Published: (2026)

LLM-Driven Multi-Turn Task-Oriented Dialogue Synthesis for Realistic Reasoning
by: Zhu, Yu, et al.
Published: (2026)

SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues
by: Kuo, Martin, et al.
Published: (2025)

Personalized LLM for Generating Customized Responses to the Same Query from Different Users
by: Zeng, Hang, et al.
Published: (2024)

LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues
by: Li, Haoyang, et al.
Published: (2025)

DIAL: Direct Iterative Adversarial Learning for Realistic Multi-Turn Dialogue Simulation
by: Zhu, Ziyi, et al.
Published: (2025)

Multi-Turn Puzzles: Evaluating Interactive Reasoning and Strategic Dialogue in LLMs
by: Badola, Kartikeya, et al.
Published: (2025)

ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreaks
by: Kim, Hyunjun, et al.
Published: (2025)

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge
by: Yang, Bo, et al.
Published: (2026)

Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue
by: Bhagtani, Kratika, et al.
Published: (2026)

Making Language Models Better Tool Learners with Execution Feedback
by: Qiao, Shuofei, et al.
Published: (2023)

CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems
by: Yang, Jingbo, et al.
Published: (2026)

CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference
by: Yu, Erxin, et al.
Published: (2024)

Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization
by: Wang, Yuhao, et al.
Published: (2025)

ECoh: Turn-level Coherence Evaluation for Multilingual Dialogues
by: Mendonça, John, et al.
Published: (2024)

KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
by: Chen, Junhao, et al.
Published: (2025)

AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue
by: Sakhawat, Adib, et al.
Published: (2026)

Discourse Diversity in Multi-Turn Empathic Dialogue
by: Zhan, Hongli, et al.
Published: (2026)

MTalk-Bench: Evaluating Speech-to-Speech Models in Multi-Turn Dialogues via Arena-style and Rubrics Protocols
by: Du, Yuhao, et al.
Published: (2025)

Evaluating Scoring Bias in LLM-as-a-Judge
by: Li, Qingquan, et al.
Published: (2025)

The Slow Drift of Support: Boundary Failures in Multi-Turn Mental Health LLM Dialogues
by: Cheng, Youyou, et al.
Published: (2026)

One Turn Too Late: Response-Aware Defense Against Hidden Malicious Intent in Multi-Turn Dialogue
by: Shen, Xinjie, et al.
Published: (2026)

Exploring Safety Alignment Evaluation of LLMs in Chinese Mental Health Dialogues via LLM-as-Judge
by: Cai, Yunna, et al.
Published: (2025)

When Metrics Disagree: Automatic Similarity vs. LLM-as-a-Judge for Clinical Dialogue Evaluation
by: Sun, Bian, et al.
Published: (2026)

SciAtlas: A Large-Scale Knowledge Graph for Automated Scientific Research
by: Qiao, Shuofei, et al.
Published: (2026)

Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
by: Hu, Yuanzhe, et al.
Published: (2025)

SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue
by: Dai, Yuqin, et al.
Published: (2026)