Copertina

JudgeBench: A Benchmark for Evaluating LLM-based Judges

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tan, Sijun, Zhuang, Siyuan, Montgomery, Kyle, Tang, William Y., Cuadron, Alejandro, Wang, Chenguang, Popa, Raluca Ada, Stoica, Ion
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Artificial Intelligence Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Budget-aware Test-time Scaling via Discriminative Verification
di: Montgomery, Kyle, et al.
Pubblicazione: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
di: Jiang, Hongchao, et al.
Pubblicazione: (2025)

Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings
di: Xu, Austin, et al.
Pubblicazione: (2025)

RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning
di: Morandi, Andrea
Pubblicazione: (2026)

Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
di: Zhou, Yilun, et al.
Pubblicazione: (2025)

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
di: Wen, Bosi, et al.
Pubblicazione: (2026)

CyclicJudge: Mitigating Judge Bias Efficiently in LLM-based Evaluation
di: Zhu, Ziyi, et al.
Pubblicazione: (2026)

RankJudge: A Multi-Turn LLM-as-a-Judge Synthetic Benchmark Generator
di: Tang, Zhenwei, et al.
Pubblicazione: (2026)

Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test
di: Zhu, Xiaoyuan, et al.
Pubblicazione: (2025)

CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems
di: Yang, Jingbo, et al.
Pubblicazione: (2026)

Who Judges the Judge? Evaluating LLM-as-a-Judge for French Medical open-ended QA
di: Belmadani, Ikram, et al.
Pubblicazione: (2026)

GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications
di: Patil, Shishir G., et al.
Pubblicazione: (2024)

JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems
di: Bellibatlu, Rohith Reddy, et al.
Pubblicazione: (2026)

Benchmarking LLM-as-a-Judge for Long-Form Output Evaluation
di: Chen, Junjie, et al.
Pubblicazione: (2026)

OR-Bench: An Over-Refusal Benchmark for Large Language Models
di: Cui, Justin, et al.
Pubblicazione: (2024)

Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation
di: Sternlicht, Noy, et al.
Pubblicazione: (2025)

LLoCO: Learning Long Contexts Offline
di: Tan, Sijun, et al.
Pubblicazione: (2024)

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
di: Thakur, Aman Singh, et al.
Pubblicazione: (2024)

MPC-Minimized Secure LLM Inference
di: Rathee, Deevashwer, et al.
Pubblicazione: (2024)

DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
di: Patel, Liana, et al.
Pubblicazione: (2025)

Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
di: Lee, Dongryeol, et al.
Pubblicazione: (2026)

Evaluating Metrics for Safety with LLM-as-Judges
di: Clegg, Kester, et al.
Pubblicazione: (2025)

MCJudgeBench: A Benchmark for Constraint-Level Judge Evaluation in Multi-Constraint Instruction Following
di: Lee, Jaeyun, et al.
Pubblicazione: (2026)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
di: Shi, Lin, et al.
Pubblicazione: (2024)

Evaluating Scoring Bias in LLM-as-a-Judge
di: Li, Qingquan, et al.
Pubblicazione: (2025)

Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation
di: Moon, Jiwon, et al.
Pubblicazione: (2025)

The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
di: Marioriyad, Arash, et al.
Pubblicazione: (2025)

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge
di: Yang, Bo, et al.
Pubblicazione: (2026)

JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
di: Shi, Zhichao, et al.
Pubblicazione: (2025)

JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
di: Jin, Jiho, et al.
Pubblicazione: (2026)

The Judge Who Never Admits: Hidden Shortcuts in LLM-based Evaluation
di: Marioriyad, Arash, et al.
Pubblicazione: (2026)

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
di: Chen, Dongping, et al.
Pubblicazione: (2024)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
di: Liu, Yixin, et al.
Pubblicazione: (2025)

Quantitative LLM Judges
di: Sahoo, Aishwarya, et al.
Pubblicazione: (2025)

MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge
di: Lee, Sua, et al.
Pubblicazione: (2026)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
di: Tong, Terry, et al.
Pubblicazione: (2025)

An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
di: Huang, Hui, et al.
Pubblicazione: (2024)

Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge
di: Zhang, Qiyuan, et al.
Pubblicazione: (2025)

LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
di: Li, Haitao, et al.
Pubblicazione: (2024)

MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models
di: Son, Guijin, et al.
Pubblicazione: (2024)