Image de couverture de livre

LLMs Judge Themselves: A Game-Theoretic Framework for Human-Aligned Evaluation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yang, Gao, Liu, Yuhang, Miao, Siyu, Liang, Xinyue, Liu, Zhengyang, Huang, Heyan
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LLMs Position Themselves as More Rational Than Humans: Emergence of AI Self-Awareness Measured Through Game Theory
par: Kim, Kyung-Hoon
Publié: (2025)

DocMEdit: Towards Document-Level Model Editing
par: Zeng, Li, et autres
Publié: (2025)

Can LLMs Explain Themselves Counterfactually?
par: Dehghanighobadi, Zahra, et autres
Publié: (2025)

When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation
par: Xu, Wenda, et autres
Publié: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
par: Liu, Yixin, et autres
Publié: (2025)

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
par: Thakur, Aman Singh, et autres
Publié: (2024)

Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
par: Tie, Guiyao, et autres
Publié: (2025)

LLMs Can Teach Themselves to Better Predict the Future
par: Turtel, Benjamin, et autres
Publié: (2025)

How Far Are We? Systematic Evaluation of LLMs vs. Human Experts in Mathematical Contest in Modeling
par: Liu, Yuhang, et autres
Publié: (2026)

Zero-Shot Detection of LLM-Generated Text via Implicit Reward Model
par: Liu, Runheng, et autres
Publié: (2026)

Language Models can Evaluate Themselves via Probability Discrepancy
par: Xia, Tingyu, et autres
Publié: (2024)

How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment
par: Huang, Heyan, et autres
Publié: (2024)

JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework
par: Liu, Fan, et autres
Publié: (2024)

Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation
par: Khalifa, Muhammad, et autres
Publié: (2026)

MCU: An Evaluation Framework for Open-Ended Game Agents
par: Zheng, Xinyue, et autres
Publié: (2023)

Enabling Weak LLMs to Judge Response Reliability via Meta Ranking
par: Liu, Zijun, et autres
Publié: (2024)

Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Games
par: Yadav, Neemesh, et autres
Publié: (2025)

Can LLMs be Good Graph Judge for Knowledge Graph Construction?
par: Huang, Haoyu, et autres
Publié: (2024)

Revisiting NLI: Towards Cost-Effective and Human-Aligned Metrics for Evaluating LLMs in Question Answering
par: Balamurali, Sai Shridhar, et autres
Publié: (2025)

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement
par: Zhou, Xiaofeng, et autres
Publié: (2025)

GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
par: Duan, Jinhao, et autres
Publié: (2024)

RAGalyst: Automated Human-Aligned Agentic Evaluation for Domain-Specific RAG
par: Gao, Joshua, et autres
Publié: (2025)

Split and Merge: Aligning Position Biases in LLM-based Evaluators
par: Li, Zongjie, et autres
Publié: (2023)

Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework
par: Xu, Kaishuai, et autres
Publié: (2025)

MVPBench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values
par: Liang, Yao, et autres
Publié: (2025)

How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments
par: Huang, Jen-tse, et autres
Publié: (2024)

Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization
par: Yao, Jiashu, et autres
Publié: (2026)

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks
par: Zhou, Xin, et autres
Publié: (2025)

EasyJudge: an Easy-to-use Tool for Comprehensive Response Evaluation of LLMs
par: Li, Yijie, et autres
Publié: (2024)

Evaluating Moral Beliefs across LLMs through a Pluralistic Framework
par: Liu, Xuelin, et autres
Publié: (2024)

Scaling Laws for Predicting Downstream Performance in LLMs
par: Chen, Yangyi, et autres
Publié: (2024)

FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs' Responsiveness to Human Feedback
par: Li, Youquan, et autres
Publié: (2024)

Criterion Validity of LLM-as-Judge for Business Outcomes in Conversational Commerce
par: Chen, Liang, et autres
Publié: (2026)

Permutation-Consensus Listwise Judging for Robust Factuality Evaluation
par: Huang, Tianyi, et autres
Publié: (2026)

ItD: Large Language Models Can Teach Themselves Induction through Deduction
par: Sun, Wangtao, et autres
Publié: (2024)

ALaRM: Align Language Models via Hierarchical Rewards Modeling
par: Lai, Yuhang, et autres
Publié: (2024)

Learning the Boundary of Solvability: Aligning LLMs to Detect Unsolvable Problems
par: Peng, Dengyun, et autres
Publié: (2025)

Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
par: Shi, Lin, et autres
Publié: (2024)

DeepSurvey-Bench: Evaluating Academic Value of Automatically Generated Scientific Survey
par: Zhang, Guo-Biao, et autres
Publié: (2026)

Reference-based Metrics Disprove Themselves in Question Generation
par: Nguyen, Bang, et autres
Publié: (2024)