Cover Image

Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Luyu, Zhang, Zeyu, Tan, Haoran, Dai, Quanyu, Yang, Hao, Dong, Zhenhua, Chen, Xu
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents
by: Tan, Haoran, et al.
Published: (2025)

MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants
by: Zhang, Zeyu, et al.
Published: (2024)

Learn to Memorize: Optimizing LLM-based Agents with Adaptive Memory Framework
by: Zhang, Zeyu, et al.
Published: (2025)

CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension
by: Li, Rui, et al.
Published: (2025)

KnowTrace: Bootstrapping Iterative Retrieval-Augmented Generation with Structured Knowledge Tracing
by: Li, Rui, et al.
Published: (2025)

Prompt and Parameter Co-Optimization for Large Language Models
by: Bo, Xiaohe, et al.
Published: (2025)

Towards Adaptive, Scalable, and Robust Coordination of LLM Agents: A Dynamic Ad-Hoc Networking Perspective
by: Li, Rui, et al.
Published: (2026)

RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems
by: Chen, Luyu, et al.
Published: (2025)

SpeechJudge: Towards Human-Level Judgment for Speech Naturalness
by: Zhang, Xueyao, et al.
Published: (2025)

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
by: Wang, Yidong, et al.
Published: (2025)

From Coarse to Fine: Self-Adaptive Hierarchical Planning for LLM Agents
by: Tan, Haoran, et al.
Published: (2026)

Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information
by: Zhang, Zeyu, et al.
Published: (2025)

MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents
by: Zhang, Zeyu, et al.
Published: (2025)

Few-shot LLM Synthetic Data with Distribution Matching
by: Ren, Jiyuan, et al.
Published: (2025)

Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era
by: Dai, Sunhao, et al.
Published: (2024)

Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models
by: Yang, Hao, et al.
Published: (2025)

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
by: Thakur, Aman Singh, et al.
Published: (2024)

HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
by: Yang, Langqi, et al.
Published: (2025)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
by: Jiang, Hongchao, et al.
Published: (2025)

From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
by: Li, Dawei, et al.
Published: (2024)

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
by: Wu, Tianhao, et al.
Published: (2024)

JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
by: Shi, Zhichao, et al.
Published: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
by: Liu, Yixin, et al.
Published: (2025)

A Survey on LLM-as-a-Judge
by: Gu, Jiawei, et al.
Published: (2024)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
by: Ye, Jiayi, et al.
Published: (2024)

How Far Are Vision-Language Models from Constructing the Real World? A Benchmark for Physical Generative Reasoning
by: Yang, Luyu, et al.
Published: (2026)

Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck
by: Zhang, Hongbin, et al.
Published: (2026)

FLAME: Factuality-Aware Alignment for Large Language Models
by: Lin, Sheng-Chieh, et al.
Published: (2024)

BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
by: Tong, Terry, et al.
Published: (2025)

Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
by: Zhang, Hongbin, et al.
Published: (2025)

Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale
by: Li, Weiyue, et al.
Published: (2026)

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
by: Yuan, Tongxin, et al.
Published: (2024)

Are We on the Right Way to Assessing LLM-as-a-Judge?
by: Feng, Yuanning, et al.
Published: (2025)

Criterion Validity of LLM-as-Judge for Business Outcomes in Conversational Commerce
by: Chen, Liang, et al.
Published: (2026)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
by: Shi, Lin, et al.
Published: (2024)

Provable Defense Framework for LLM Jailbreaks via Noise-Augumented Alignment
by: Cheng, Zehua, et al.
Published: (2026)

Beyond Single-Sentence Prompts: Upgrading Value Alignment Benchmarks with Dialogues and Stories
by: Zhang, Yazhou, et al.
Published: (2025)

Understanding Layer Significance in LLM Alignment
by: Shi, Guangyuan, et al.
Published: (2024)

EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models
by: Mohammadi, Hadi, et al.
Published: (2025)