Copertina

Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Spiliopoulou, Evangelia, Fogliato, Riccardo, Burnsky, Hanna, Soliman, Tamer, Ma, Jie, Horwood, Graham, Ballesteros, Miguel
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generation
di: Riaz, Haris, et al.
Pubblicazione: (2025)

General Purpose Verification for Chain of Thought Prompting
di: Vacareanu, Robert, et al.
Pubblicazione: (2024)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
di: Shi, Lin, et al.
Pubblicazione: (2024)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
di: Yang, Jinming, et al.
Pubblicazione: (2026)

Detecting Training Data of Large Language Models via Expectation Maximization
di: Kim, Gyuwan, et al.
Pubblicazione: (2024)

Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge
di: Cantini, Riccardo, et al.
Pubblicazione: (2025)

Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models
di: Kumar, Shachi H, et al.
Pubblicazione: (2024)

Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles
di: Truong, Kimberly Le, et al.
Pubblicazione: (2025)

TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notes
di: Shah, Raj Sanjay, et al.
Pubblicazione: (2025)

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
di: Lai, Peng, et al.
Pubblicazione: (2026)

Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge
di: Zhou, Xiaolin, et al.
Pubblicazione: (2026)

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge
di: Fujinuma, Yoshinari
Pubblicazione: (2025)

Active Evaluation Acquisition for Efficient LLM Benchmarking
di: Li, Yang, et al.
Pubblicazione: (2024)

SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks
di: Safarzadeh, Mohammadtaher, et al.
Pubblicazione: (2026)

Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck
di: Zhang, Hongbin, et al.
Pubblicazione: (2026)

Self-Preference Bias in LLM-as-a-Judge
di: Wataoka, Koki, et al.
Pubblicazione: (2024)

Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
di: Xu, Wenda, et al.
Pubblicazione: (2024)

M-Prometheus: A Suite of Open Multilingual LLM Judges
di: Pombal, José, et al.
Pubblicazione: (2025)

A Survey on LLM-as-a-Judge
di: Gu, Jiawei, et al.
Pubblicazione: (2024)

SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification
di: Yoon, Kanghoon, et al.
Pubblicazione: (2025)

SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning
di: Chen, Jiaqi, et al.
Pubblicazione: (2025)

A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness
di: Schwinn, Leo, et al.
Pubblicazione: (2026)

Training Language Models to Win Debates with Self-Play Improves Judge Accuracy
di: Arnesen, Samuel, et al.
Pubblicazione: (2024)

GSM-SEM: Benchmark and Framework for Generating Semantically Variant Augmentations
di: Singh, Jyotika, et al.
Pubblicazione: (2026)

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
di: Wu, Tianhao, et al.
Pubblicazione: (2024)

Do Before You Judge: Self-Reference as a Pathway to Better LLM Evaluation
di: Lin, Wei-Hsiang, et al.
Pubblicazione: (2025)

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
di: Wang, Yidong, et al.
Pubblicazione: (2025)

Improving LLM Reasoning through Interpretable Role-Playing Steering
di: Wang, Anyi, et al.
Pubblicazione: (2025)

Improving LLM Code Reasoning via Semantic Equivalence Self-Play with Formal Verification
di: Barone, Antonio Valerio Miceli, et al.
Pubblicazione: (2026)

Evaluating Metrics for Safety with LLM-as-Judges
di: Clegg, Kester, et al.
Pubblicazione: (2025)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
di: Tong, Terry, et al.
Pubblicazione: (2025)

Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement
di: Han, Steve, et al.
Pubblicazione: (2025)

JudgeBench: A Benchmark for Evaluating LLM-based Judges
di: Tan, Sijun, et al.
Pubblicazione: (2024)

BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM Evaluation
di: Gisserot-Boukhlef, Hippolyte, et al.
Pubblicazione: (2026)

LLM-as-a-Judge for Time Series Explanations
di: Sivalingam, Preetham, et al.
Pubblicazione: (2026)

Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Style
di: Han, Xueran, et al.
Pubblicazione: (2025)

The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
di: Marioriyad, Arash, et al.
Pubblicazione: (2025)

RPA-Check: A Multi-Stage Automated Framework for Evaluating Dynamic LLM-based Role-Playing Agents
di: Rosati, Riccardo, et al.
Pubblicazione: (2026)

Self-Assessment Tests are Unreliable Measures of LLM Personality
di: Gupta, Akshat, et al.
Pubblicazione: (2023)

Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation
di: Cantini, Riccardo, et al.
Pubblicazione: (2024)