Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910951427538944 |
|---|---|
| author | Chen, Luyu Zhang, Zeyu Tan, Haoran Dai, Quanyu Yang, Hao Dong, Zhenhua Chen, Xu |
| author_facet | Chen, Luyu Zhang, Zeyu Tan, Haoran Dai, Quanyu Yang, Hao Dong, Zhenhua Chen, Xu |
| contents | LLMs have emerged as powerful evaluators in the LLM-as-a-Judge paradigm, offering significant efficiency and flexibility compared to human judgments. However, previous methods primarily rely on single-point evaluations, overlooking the inherent diversity and uncertainty in human evaluations. This approach leads to information loss and decreases the reliability of evaluations. To address this limitation, we propose a novel training framework that explicitly aligns the LLM-generated judgment distribution with empirical human distributions. Specifically, we propose a distributional alignment objective based on KL divergence, combined with an auxiliary cross-entropy regularization to stabilize the training process. Furthermore, considering that empirical distributions may derive from limited human annotations, we incorporate adversarial training to enhance model robustness against distribution perturbations. Extensive experiments across various LLM backbones and evaluation tasks demonstrate that our framework significantly outperforms existing closed-source LLMs and conventional single-point alignment methods, with improved alignment quality, evaluation accuracy, and robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_12301 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge Chen, Luyu Zhang, Zeyu Tan, Haoran Dai, Quanyu Yang, Hao Dong, Zhenhua Chen, Xu Artificial Intelligence Computation and Language LLMs have emerged as powerful evaluators in the LLM-as-a-Judge paradigm, offering significant efficiency and flexibility compared to human judgments. However, previous methods primarily rely on single-point evaluations, overlooking the inherent diversity and uncertainty in human evaluations. This approach leads to information loss and decreases the reliability of evaluations. To address this limitation, we propose a novel training framework that explicitly aligns the LLM-generated judgment distribution with empirical human distributions. Specifically, we propose a distributional alignment objective based on KL divergence, combined with an auxiliary cross-entropy regularization to stabilize the training process. Furthermore, considering that empirical distributions may derive from limited human annotations, we incorporate adversarial training to enhance model robustness against distribution perturbations. Extensive experiments across various LLM backbones and evaluation tasks demonstrate that our framework significantly outperforms existing closed-source LLMs and conventional single-point alignment methods, with improved alignment quality, evaluation accuracy, and robustness. |
| title | Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2505.12301 |