Cover Image

Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgments

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Yuran, Mohamud, Jama Hussein, Sun, Chongren, Wu, Di, Boulet, Benoit
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OnionEval: An Unified Evaluation of Fact-conflicting Hallucination for Small-Large Language Models
by: Sun, Chongren, et al.
Published: (2025)

An empirical study of task and feature correlations in the reuse of pre-trained models
by: Mohamud, Jama Hussein, et al.
Published: (2025)

Self-Routing: Parameter-Free Expert Routing from Hidden States
by: Mohamud, Jama Hussein, et al.
Published: (2026)

STEMS: Spatial-Temporal Enhanced Safe Multi-Agent Coordination for Building Energy Management
by: Zhang, Huiliang, et al.
Published: (2025)

Meta-Reinforcement Learning for Building Energy Management System
by: Zhang, Huiliang, et al.
Published: (2022)

Goal-Conditioned Data Augmentation for Offline Reinforcement Learning
by: Huang, Xingshuai, et al.
Published: (2024)

An Online Spatial-Temporal Graph Trajectory Planner for Autonomous Vehicles
by: Samiuddin, Jilan, et al.
Published: (2024)

Leveraging Multivariate Long-Term History Representation for Time Series Forecasting
by: Zhang, Huiliang, et al.
Published: (2025)

Who's Your Judge? On the Detectability of LLM-Generated Judgments
by: Li, Dawei, et al.
Published: (2025)

A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement
by: Li, Yuran, et al.
Published: (2026)

When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
by: Yu, Fangyi
Published: (2025)

Robot Policy Learning with Temporal Optimal Transport Reward
by: Fu, Yuwei, et al.
Published: (2024)

PEMANT: Persona-Enriched Multi-Agent Negotiation for Travel
by: Sun, Yuran, et al.
Published: (2026)

Nearest Neighbor Multivariate Time Series Forecasting
by: Zhang, Huiliang, et al.
Published: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
by: Liu, Yixin, et al.
Published: (2025)

Unifying Temporal and Structural Credit Assignment in LLM-Based Multi-Agent Prompt Optimization
by: Li, Wenwu, et al.
Published: (2026)

Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge
by: Chen, Luyu, et al.
Published: (2025)

ForeSci: Evaluating LLM Agents for Forward-Looking AI Research Judgment
by: Tian, Qiuyu, et al.
Published: (2026)

Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction
by: Chen, Xi, et al.
Published: (2025)

Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judge
by: Yang, Wei, et al.
Published: (2026)

Multi-Agent Debate for LLM Judges with Adaptive Stability Detection
by: Hu, Tianyu, et al.
Published: (2025)

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
by: Thakur, Aman Singh, et al.
Published: (2024)

FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning
by: Fu, Yuwei, et al.
Published: (2024)

EasyJudge: an Easy-to-use Tool for Comprehensive Response Evaluation of LLMs
by: Li, Yijie, et al.
Published: (2024)

SpeechJudge: Towards Human-Level Judgment for Speech Naturalness
by: Zhang, Xueyao, et al.
Published: (2025)

Retrieval- and Argumentation-Enhanced Multi-Agent LLMs for Judgmental Forecasting (Extended Version with Supplementary Material)
by: Gorur, Deniz, et al.
Published: (2025)

Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment
by: Wu, Shunyu, et al.
Published: (2025)

The Necessity of a Unified Framework for LLM-Based Agent Evaluation
by: Zhu, Pengyu, et al.
Published: (2026)

RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following
by: Pan, Tianjun, et al.
Published: (2026)

Agent-as-a-Judge: Evaluate Agents with Agents
by: Zhuge, Mingchen, et al.
Published: (2024)

Beyond Surface Judgments: Human-Grounded Risk Evaluation of LLM-Generated Disinformation
by: Xu, Zonghuan, et al.
Published: (2026)

JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework
by: Liu, Fan, et al.
Published: (2024)

Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges
by: Koo, Hamin, et al.
Published: (2025)

Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelines
by: Soumik, Sadman Kabir
Published: (2026)

Judging the Judges: Human Validation of Multi-LLM Evaluation for High-Quality K--12 Science Instructional Materials
by: He, Peng, et al.
Published: (2026)

CollabEval: Enhancing LLM-as-a-Judge via Multi-Agent Collaboration
by: Qian, Yiyue, et al.
Published: (2026)

JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
by: Shi, Zhichao, et al.
Published: (2025)

LLMs Judge Themselves: A Game-Theoretic Framework for Human-Aligned Evaluation
by: Yang, Gao, et al.
Published: (2025)

Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environments
by: Boulet, Timothé, et al.
Published: (2025)

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
by: Wu, Tianhao, et al.
Published: (2024)