Cover Image

Neither Valid nor Reliable? Investigating the Use of LLMs as Judges

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chehbouni, Khaoula, Haddou, Mohammed, Cheung, Jackie Chi Kit, Farnadi, Golnoosh
Format:	Preprint
Published:	2025
Subjects:	Computation and Language I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
by: Saji, Alan, et al.
Published: (2025)

MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMs
by: Bouchekif, Abdessalam, et al.
Published: (2026)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
by: Ashuach, Tomer, et al.
Published: (2025)

RedHerring Attack: Testing the Reliability of Attack Detection
by: Rusert, Jonathan
Published: (2025)

Large-scale cloze evaluation reveals that token prediction tasks are neither lexically nor semantically aligned
by: Jacobs, Cassandra L., et al.
Published: (2024)

Investigating the Impact of Text Summarization on Topic Modeling
by: Khandelwal, Trishia
Published: (2024)

Constructing Benchmarks and Interventions for Combating Hallucinations in LLMs
by: Simhi, Adi, et al.
Published: (2024)

EnDive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models
by: Gupta, Abhay, et al.
Published: (2025)

LLMs Are Not Scorers: Rethinking MT Evaluation with Generation-Based Methods
by: Cui, Hyang
Published: (2025)

MIRIAD: Augmenting LLMs with millions of medical query-response pairs
by: Zheng, Qinyue, et al.
Published: (2025)

RAG-Optimized Tibetan Tourism LLMs: Enhancing Accuracy and Personalization
by: Qi, Jinhu, et al.
Published: (2024)

ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs
by: Simhi, Adi, et al.
Published: (2025)

Fine-Tuning LLMs with Fine-Grained Human Feedback on Text Spans
by: CH-Wang, Sky, et al.
Published: (2025)

LLMs and the Human Condition
by: Wallis, Peter
Published: (2024)

Generator-Guided Crowd Reaction Assessment
by: Ghosh, Sohom, et al.
Published: (2024)

Overview of the Sensemaking Task at the ELOQUENT 2025 Lab: LLMs as Teachers, Students and Evaluators
by: Šindelář, Pavel, et al.
Published: (2025)

Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER
by: Ewais, Ahmed, et al.
Published: (2026)

MALT: Mechanistic Ablation of Lossy Translation in LLMs for a Low-Resource Language: Urdu
by: Bajwa, Taaha Saleem
Published: (2025)

Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer
by: Simhi, Adi, et al.
Published: (2025)

Temporal Referential Consistency: Do LLMs Favor Sequences Over Absolute Time References?
by: Bajpai, Ashutosh, et al.
Published: (2025)

Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection
by: Stacey, Joe, et al.
Published: (2025)

A Case Study of Cross-Lingual Zero-Shot Generalization for Classical Languages in LLMs
by: Akavarapu, V. S. D. S. Mahesh, et al.
Published: (2025)

Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs
by: Cui, Wanyun, et al.
Published: (2025)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
by: Peters, Sydney, et al.
Published: (2025)

All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens
by: Mamidanna, Siddarth, et al.
Published: (2025)

OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs
by: Wang, Yuxia, et al.
Published: (2024)

EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers
by: Jiang, Yilin, et al.
Published: (2025)

Improving LLMs with a knowledge from databases
by: Máša, Petr
Published: (2025)

Assessing RAG and HyDE on 1B vs. 4B-Parameter Gemma LLMs for Personal Assistants Integretion
by: Sorstkins, Andrejs
Published: (2025)

Layer-Aware Embedding Fusion for LLMs in Text Classifications
by: Gwak, Jiho, et al.
Published: (2025)

Syntactic Blind Spots: How Misalignment Leads to LLMs Mathematical Errors
by: Williamson, Dane, et al.
Published: (2025)

ML-Promise: A Multilingual Dataset for Corporate Promise Verification
by: Seki, Yohei, et al.
Published: (2024)

Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
by: Collado-Montañez, Jaime, et al.
Published: (2025)

SeLeRoSa: Sentence-Level Romanian Satire Detection Dataset
by: Smădu, Răzvan-Alexandru, et al.
Published: (2025)

Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation
by: Resck, Lucas, et al.
Published: (2026)

HalluScan: A Systematic Benchmark for Detecting and Mitigating Hallucinations in Instruction-Following LLMs
by: Cherif, Ahmed
Published: (2026)

The Syntax of qulk-clauses in Yemeni Ibbi Arabic: A Minimalist Approach
by: Albadani, Zubaida Mohammed, et al.
Published: (2025)

Current LLMs still cannot 'talk much' about grammar modules: Evidence from syntax
by: Shormani, Mohammed Q., et al.
Published: (2026)

Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking
by: Zhang, Liangliang, et al.
Published: (2025)

Generative linguistics contribution to artificial intelligence: Where this contribution lies?
by: Shormani, Mohammed Q.
Published: (2024)