Cover Image

The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Marioriyad, Arash, Rohban, Mohammad Hossein, Baghshah, Mahdieh Soleymani
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Judge Who Never Admits: Hidden Shortcuts in LLM-based Evaluation
by: Marioriyad, Arash, et al.
Published: (2026)

Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing
by: Marioriyad, Arash, et al.
Published: (2026)

Why Settle for Mid: A Probabilistic Viewpoint to Spatial Relationship Alignment in Text-to-image Models
by: Rezaei, Parham, et al.
Published: (2025)

Diffusion Beats Autoregressive: An Evaluation of Compositional Generation in Text-to-Image Models
by: Marioriyad, Arash, et al.
Published: (2024)

Unspoken Hints: Accuracy Without Acknowledgement in LLM Reasoning
by: Marioriyad, Arash, et al.
Published: (2025)

Large Language Models for Scientific Idea Generation: A Creativity-Centered Survey
by: Shahhosseini, Fatemeh, et al.
Published: (2025)

Attention Overlap Is Responsible for The Entity Missing Problem in Text-to-image Diffusion Models!
by: Marioriyad, Arash, et al.
Published: (2024)

Evaluating the Evaluators: Metrics for Compositional Text-to-Image Generation
by: Kasaei, Seyed Amir, et al.
Published: (2025)

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP
by: Abbasi, Reza, et al.
Published: (2024)

CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration
by: Kasaei, Seyed Amir, et al.
Published: (2025)

No Concept Left Behind: Test-Time Optimization for Compositional Text-to-Image Generation
by: Sameti, Mohammad Hossein, et al.
Published: (2025)

Hidden Meanings in Plain Sight: RebusBench for Evaluating Cognitive Visual Reasoning
by: Kasaei, Seyed Amir, et al.
Published: (2026)

Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Models
by: Abbasi, Reza, et al.
Published: (2024)

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
by: Ghahroodi, Omid, et al.
Published: (2024)

Infinity and Beyond: Compositional Alignment in VAR and Diffusion T2I Models
by: Shahabadi, Hossein, et al.
Published: (2025)

VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization
by: Paqaleh, Mohammad Mahdi Samiei, et al.
Published: (2025)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
by: Shi, Lin, et al.
Published: (2024)

Persian Musical Instruments Classification Using Polyphonic Data Augmentation
by: Esfangereh, Diba Hadi, et al.
Published: (2025)

Persian MusicGen: A Large-Scale Dataset and Culturally-Aware Generative Model for Persian Music
by: Sameti, Mohammad Hossein, et al.
Published: (2026)

ELAB: Extensive LLM Alignment Benchmark in Persian Language
by: Pourbahman, Zahra, et al.
Published: (2025)

CyclicJudge: Mitigating Judge Bias Efficiently in LLM-based Evaluation
by: Zhu, Ziyi, et al.
Published: (2026)

Analyzing CLIP's Performance Limitations in Multi-Object Scenarios: A Controlled High-Resolution Study
by: Abbasi, Reza, et al.
Published: (2025)

CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation
by: Abbasi, Reza, et al.
Published: (2025)

Evaluating Scoring Bias in LLM-as-a-Judge
by: Li, Qingquan, et al.
Published: (2025)

Self-Preference Bias in LLM-as-a-Judge
by: Wataoka, Koki, et al.
Published: (2024)

Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detection
by: Zohrabi, Reihaneh, et al.
Published: (2025)

GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models
by: Abdollahi, Ali, et al.
Published: (2024)

Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning
by: Dijujin, Negin Hashemi, et al.
Published: (2025)

Checklist Engineering Empowers Multilingual LLM Judges
by: Mohammadkhani, Mohammad Ghiasvand, et al.
Published: (2025)

Assistant-Guided Mitigation of Teacher Preference Bias in LLM-as-a-Judge
by: Liu, Zhuo, et al.
Published: (2025)

Who Judges the Judge? Evaluating LLM-as-a-Judge for French Medical open-ended QA
by: Belmadani, Ikram, et al.
Published: (2026)

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge
by: Yang, Bo, et al.
Published: (2026)

Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges
by: Tapwal, Riya, et al.
Published: (2026)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
by: Yang, Jinming, et al.
Published: (2026)

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge
by: Fujinuma, Yoshinari
Published: (2025)

Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge
by: Zhou, Xiaolin, et al.
Published: (2026)

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
by: Lai, Peng, et al.
Published: (2026)

JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems
by: Bellibatlu, Rohith Reddy, et al.
Published: (2026)

Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization
by: Zhou, Hongli, et al.
Published: (2026)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
by: Tong, Terry, et al.
Published: (2025)