Copertina

Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applications

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cao, Hongliu, Driouich, Ilias, Singh, Robin, Thomas, Eoin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Diverse And Private Synthetic Datasets Generation for RAG evaluation: A multi-agent framework
di: Driouich, Ilias, et al.
Pubblicazione: (2025)

Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
di: Cao, Hongliu, et al.
Pubblicazione: (2026)

LLM-as-a-qualitative-judge: automating error analysis in natural language generation
di: Chirkova, Nadezhda, et al.
Pubblicazione: (2025)

Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universality
di: Cao, Hongliu
Pubblicazione: (2025)

Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmark
di: Cao, Hongliu
Pubblicazione: (2024)

From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
di: Li, Dawei, et al.
Pubblicazione: (2024)

When Wording Steers the Evaluation: Framing Bias in LLM judges
di: Hwang, Yerin, et al.
Pubblicazione: (2026)

From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
di: Stephan, Andreas, et al.
Pubblicazione: (2024)

When LLMs Imagine People: A Human-Centered Persona Brainstorm Audit for Bias and Fairness in Creative Applications
di: Cao, Hongliu, et al.
Pubblicazione: (2026)

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
di: Yuan, Tongxin, et al.
Pubblicazione: (2024)

Preference Leakage: A Contamination Problem in LLM-as-a-judge
di: Li, Dawei, et al.
Pubblicazione: (2025)

RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning
di: Morandi, Andrea
Pubblicazione: (2026)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
di: Shi, Lin, et al.
Pubblicazione: (2024)

JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
di: Shi, Zhichao, et al.
Pubblicazione: (2025)

Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards
di: Wei, Xiaolong, et al.
Pubblicazione: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
di: Jiang, Hongchao, et al.
Pubblicazione: (2025)

A Survey on LLM-as-a-Judge
di: Gu, Jiawei, et al.
Pubblicazione: (2024)

Evaluating Metrics for Safety with LLM-as-Judges
di: Clegg, Kester, et al.
Pubblicazione: (2025)

Balanced Accuracy: The Right Metric for Evaluating LLM Judges -- Explained through Youden's J statistic
di: Collot, Stephane, et al.
Pubblicazione: (2025)

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
di: Wang, Yidong, et al.
Pubblicazione: (2025)

The "LLM World of Words" English free association norms generated by large language models
di: Abramski, Katherine, et al.
Pubblicazione: (2024)

LLM-as-a-Judge for Time Series Explanations
di: Sivalingam, Preetham, et al.
Pubblicazione: (2026)

LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluation
di: Enguehard, Joseph, et al.
Pubblicazione: (2025)

Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement
di: Han, Steve, et al.
Pubblicazione: (2025)

The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation
di: Gouvert, Olivier, et al.
Pubblicazione: (2025)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
di: Tong, Terry, et al.
Pubblicazione: (2025)

VERT: Reliable LLM Judges for Radiology Report Evaluation
di: Bologna, Federica, et al.
Pubblicazione: (2026)

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
di: Ye, Jiayi, et al.
Pubblicazione: (2024)

Are We on the Right Way to Assessing LLM-as-a-Judge?
di: Feng, Yuanning, et al.
Pubblicazione: (2025)

Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring
di: Dussert, Jehanne
Pubblicazione: (2026)

Local Model Reconstruction Attacks in Federated Learning and their Uses
di: Driouich, Ilias, et al.
Pubblicazione: (2022)

Emergent Convergence in Multi-Agent LLM Annotation
di: Parfenova, Angelina, et al.
Pubblicazione: (2025)

MIRIX: Multi-Agent Memory System for LLM-Based Agents
di: Wang, Yu, et al.
Pubblicazione: (2025)

An evaluation of LLM code generation capabilities through graded exercises
di: Jiménez, Álvaro Barbero
Pubblicazione: (2024)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
di: Tan, Sijun, et al.
Pubblicazione: (2024)

How good is my story? Towards quantitative metrics for evaluating LLM-generated XAI narratives
di: Ichmoukhamedov, Timour, et al.
Pubblicazione: (2024)

Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
di: Saha, Swarnadeep, et al.
Pubblicazione: (2025)

M-Prometheus: A Suite of Open Multilingual LLM Judges
di: Pombal, José, et al.
Pubblicazione: (2025)

Criterion Validity of LLM-as-Judge for Business Outcomes in Conversational Commerce
di: Chen, Liang, et al.
Pubblicazione: (2026)

Think-J: Learning to Think for Generative LLM-as-a-Judge
di: Huang, Hui, et al.
Pubblicazione: (2025)