Cover Image

Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Roytburg, Dani, Bozoukov, Matthew, Nguyen, Matthew, Barzdukas, Jou, Puig-Hall, Mackenzie, Oozeer, Narmeen
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
by: Roytburg, Dani, et al.
Published: (2025)

Minimal and Mechanistic Conditions for Behavioral Self-Awareness in LLMs
by: Bozoukov, Matthew, et al.
Published: (2025)

Understanding and Mitigating Dataset Corruption in LLM Steering
by: Anderson, Cullen, et al.
Published: (2026)

Beyond Linear Steering: Unified Multi-Attribute Control for Language Models
by: Oozeer, Narmeen, et al.
Published: (2025)

Approximating Human Preferences Using a Multi-Judge Learned System
by: Sprejer, Eitán, et al.
Published: (2025)

Bilinear Convolution Decomposition for Causal RL Interpretability
by: Oozeer, Narmeen, et al.
Published: (2024)

Sanity Checks for Explanation Uncertainty
by: Valdenegro-Toro, Matias, et al.
Published: (2024)

Sanity Checks for Agentic Data Science
by: Rewolinski, Zachary T., et al.
Published: (2026)

Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test
by: Hedström, Anna, et al.
Published: (2024)

Sanity Checks for Long-Form Hallucination Detection
by: Zollicoffer, Geigh, et al.
Published: (2026)

A Fresh Look at Sanity Checks for Saliency Maps
by: Hedström, Anna, et al.
Published: (2024)

Sanity Checking Causal Representation Learning on a Simple Real-World System
by: Gamella, Juan L., et al.
Published: (2025)

Measuring Reasoning Trace Legibility: Can Those Who Understand Teach?
by: Roytburg, Dani, et al.
Published: (2026)

Self-Verification Dilemma: Experience-Driven Suppression of Overused Checking in LLM Reasoning
by: Long, Quanyu, et al.
Published: (2026)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
by: Yang, Jinming, et al.
Published: (2026)

Towards Acyclic Preference Evaluation of Language Models via Multiple Evaluators
by: Hu, Zhengyu, et al.
Published: (2024)

Spectral Superposition: A Theory of Feature Geometry
by: Ivanov, Georgi, et al.
Published: (2026)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
by: Liu, Yixin, et al.
Published: (2025)

Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
by: Rahman, Subhey Sadi, et al.
Published: (2025)

Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation
by: Wang, Yiming, et al.
Published: (2024)

Post-training an LLM for RAG? Train on Self-Generated Demonstrations
by: Finlayson, Matthew, et al.
Published: (2025)

First is Not Really Better Than Last: Evaluating Layer Choice and Aggregation Strategies in Language Model Data Influence Estimation
by: Vitel, Dmytro, et al.
Published: (2025)

InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoning
by: Yang, Matthew Y. R., et al.
Published: (2026)

Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences
by: Karlekar, Sweta, et al.
Published: (2026)

LLM-Based Adversarial Persuasion Attacks on Fact-Checking Systems
by: Leite, João A., et al.
Published: (2026)

Self-Consistency Preference Optimization
by: Prasad, Archiki, et al.
Published: (2024)

Enhancing LLM Evaluations: The Garbling Trick
by: Bradley, William F.
Published: (2024)

Survey on Evaluation of LLM-based Agents
by: Yehudai, Asaf, et al.
Published: (2025)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
by: Liu, Yinhong, et al.
Published: (2024)

Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking
by: Akhtar, Mubashara, et al.
Published: (2024)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
by: Gupta, Taneesh, et al.
Published: (2025)

A Dataset for Evaluating LLM-based Evaluation Functions for Research Question Extraction Task
by: Fujisaki, Yuya, et al.
Published: (2024)

Towards Multilingual LLM Evaluation for European Languages
by: Thellmann, Klaudia, et al.
Published: (2024)

Evaluating the Relevance of Uncertainty Estimators for LLM Hallucination
by: Agnimo, Yedidia, et al.
Published: (2026)

Code Comprehension then Auditing for Unsupervised LLM Evaluation
by: Patel, Bhrij, et al.
Published: (2024)

FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs
by: Sawczyn, Albert, et al.
Published: (2025)

Evaluation is All You Need: Strategic Overclaiming of LLM Reasoning Capabilities Through Evaluation Design
by: Sun, Lin, et al.
Published: (2025)

Advancing LLM Reasoning Generalists with Preference Trees
by: Yuan, Lifan, et al.
Published: (2024)

TSO: Self-Training with Scaled Preference Optimization
by: Chen, Kaihui, et al.
Published: (2024)