Cover Image

Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ma, Chiyu, Zhang, Enpei, Zhao, Yilun, Liu, Wenjun, Jia, Yaning, Qing, Peijun, Shi, Lin, Cohan, Arman, Yan, Yujun, Vosoughi, Soroush
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
by: Shi, Lin, et al.
Published: (2024)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
by: Liu, Yixin, et al.
Published: (2025)

LimRank: Less is More for Reasoning-Intensive Information Reranking
by: Song, Tingyu, et al.
Published: (2025)

KCES: Training-Free Defense for Robust Graph Neural Networks via Kernel Complexity
by: Jia, Yaning, et al.
Published: (2025)

SoundMind: RL-Incentivized Logic Reasoning for Audio-Language Models
by: Diao, Xingjian, et al.
Published: (2025)

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
by: Ye, Jiayi, et al.
Published: (2024)

What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning
by: Jia, Yaning, et al.
Published: (2025)

Achieving Domain-Independent Certified Robustness via Knowledge Continuity
by: Sun, Alan, et al.
Published: (2024)

Interpretable Image Classification with Adaptive Prototype-based Vision Transformers
by: Ma, Chiyu, et al.
Published: (2024)

The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
by: Marioriyad, Arash, et al.
Published: (2025)

Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelines
by: Soumik, Sadman Kabir
Published: (2026)

Scaled Supervision is an Implicit Lipschitz Regularizer
by: Ouyang, Zhongyu, et al.
Published: (2025)

CyclicJudge: Mitigating Judge Bias Efficiently in LLM-based Evaluation
by: Zhu, Ziyi, et al.
Published: (2026)

Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
by: Wang, Leyao, et al.
Published: (2026)

Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
by: Zhou, Yilun, et al.
Published: (2025)

Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
by: Liu, Yixin, et al.
Published: (2026)

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
by: Wang, Yidong, et al.
Published: (2025)

AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality
by: Qing, Peijun, et al.
Published: (2024)

Evaluating Scoring Bias in LLM-as-a-Judge
by: Li, Qingquan, et al.
Published: (2025)

Self-Preference Bias in LLM-as-a-Judge
by: Wataoka, Koki, et al.
Published: (2024)

Bias and Uncertainty in LLM-as-a-Judge Estimation
by: Fiedler, James
Published: (2026)

Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?
by: Kim, Jane Paik
Published: (2026)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
by: Yang, Jinming, et al.
Published: (2026)

Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge
by: Xu, Yuzheng, et al.
Published: (2026)

ANCHOR: Branch-Point Data Generation for GUI Agents
by: Wei, Jinbiao, et al.
Published: (2026)

SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
by: Liu, Hongjun, et al.
Published: (2025)

MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search
by: Hu, Yunhai, et al.
Published: (2025)

Table-R1: Inference-Time Scaling for Table Reasoning
by: Yang, Zheyuan, et al.
Published: (2025)

Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers
by: Zhao, Yilun, et al.
Published: (2025)

BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
by: Lai, Peng, et al.
Published: (2026)

Beyond Consensus: Mitigating the Agreeableness Bias in LLM Judge Evaluations
by: Jain, Suryaansh, et al.
Published: (2025)

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering
by: Zhao, Zixiao, et al.
Published: (2026)

Disordered-DABS: A Benchmark for Dynamic Aspect-Based Summarization in Disordered Texts
by: Guo, Xiaobo, et al.
Published: (2024)

MODABS: Multi-Objective Learning for Dynamic Aspect-Based Summarization
by: Guo, Xiaobo, et al.
Published: (2024)

Serial Position Effects of Large Language Models
by: Guo, Xiaobo, et al.
Published: (2024)

Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making
by: Plecko, Drago, et al.
Published: (2024)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
by: Tong, Terry, et al.
Published: (2025)

Doing More with Less.
by: Wagenveld, Linda M.
Published: (1987)

AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling
by: Miao, Yongliang, et al.
Published: (2026)

Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
by: Zhang, Siyue, et al.
Published: (2025)