Cover Image

Benchmarking LLMs' Judgments with No Gold Standard

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Shengwei, Lu, Yuxuan, Schoenebeck, Grant, Kong, Yuqing
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Eliciting Informative Text Evaluations with Large Language Models
by: Lu, Yuxuan, et al.
Published: (2024)

Jailbreaking LLMs via Calibration
by: Lu, Yuxuan, et al.
Published: (2026)

Principled Evaluation with Human Labels: One Rater at a Time and Rater Equivalence
by: Resnick, Paul, et al.
Published: (2021)

Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms
by: Xu, Shengwei, et al.
Published: (2024)

Evaluating the Correctness of Inference Patterns Used by LLMs for Judgment
by: Chen, Lu, et al.
Published: (2024)

AppellateGen: A Benchmark for Appellate Legal Judgment Generation
by: Yang, Hongkun, et al.
Published: (2026)

Measuring and Reducing LLM Hallucination without Gold-Standard Answers
by: Wei, Jiaheng, et al.
Published: (2024)

GPT-4o as the Gold Standard: A Scalable and General Purpose Approach to Filter Language Model Pretraining Data
by: Zhang, Jifan, et al.
Published: (2024)

Ad Insertion in LLM-Generated Responses
by: Xu, Shengwei, et al.
Published: (2026)

Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgments
by: Li, Jiaxi, et al.
Published: (2025)

Demystifying Long Chain-of-Thought Reasoning in LLMs
by: Yeo, Edward, et al.
Published: (2025)

Benchmarking and Understanding Compositional Relational Reasoning of LLMs
by: Ni, Ruikang, et al.
Published: (2024)

A Korean Legal Judgment Prediction Dataset for Insurance Disputes
by: Kwak, Alice Saebom, et al.
Published: (2024)

Curiosity-Driven LLM-as-a-judge for Personalized Creative Judgment
by: Kumar, Vanya Bannihatti, et al.
Published: (2025)

TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs
by: Gu, Yuxuan, et al.
Published: (2025)

An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs
by: Zhang, Hengran, et al.
Published: (2024)

Bridging Human and LLM Judgments: Understanding and Narrowing the Gap
by: Polo, Felipe Maia, et al.
Published: (2025)

PATENTWRITER: A Benchmarking Study for Patent Drafting with LLMs
by: Shomee, Homaira Huda, et al.
Published: (2025)

BioCoref: Benchmarking Biomedical Coreference Resolution with LLMs
by: Salem, Nourah M, et al.
Published: (2025)

Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs
by: Sun, Huaman, et al.
Published: (2023)

BONUS! Maximizing Surprise
by: Huang, Zhihuan, et al.
Published: (2021)

tinyBenchmarks: evaluating LLMs with fewer examples
by: Polo, Felipe Maia, et al.
Published: (2024)

Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing
by: Xian, Ruicheng, et al.
Published: (2025)

Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
by: Kim, Jeonghye, et al.
Published: (2026)

How Many Human Judgments Are Enough? Feasibility Limits of Human Preference Evaluation
by: Lee, Wilson Y.
Published: (2026)

Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments
by: Linhardt, Lorenz, et al.
Published: (2025)

From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench
by: Shi, Weikang, et al.
Published: (2026)

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
by: Jiang, Huiqiang, et al.
Published: (2023)

AutoTriton: Automatic Triton Programming with Reinforcement Learning in LLMs
by: Li, Shangzhan, et al.
Published: (2025)

LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs
by: Zhou, Yujun, et al.
Published: (2024)

Evaluating LLMs' Multilingual Capabilities for Bengali: Benchmark Creation and Performance Analysis
by: Bhowmik, Shimanto, et al.
Published: (2025)

XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning
by: Zhang, Zhihan, et al.
Published: (2025)

Fighting Spurious Correlations in Text Classification via a Causal Learning Perspective
by: Zhou, Yuqing, et al.
Published: (2024)

Benchmarking Hindi LLMs: A New Suite of Datasets and a Comparative Analysis
by: Kamath, Anusha, et al.
Published: (2025)

WirelessMathBench: A Mathematical Modeling Benchmark for LLMs in Wireless Communications
by: Li, Xin, et al.
Published: (2025)

MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs
by: Lee, Junhyeok, et al.
Published: (2026)

Learn and Unlearn: Addressing Misinformation in Multilingual LLMs
by: Lu, Taiming, et al.
Published: (2024)

Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
by: Benhenda, Mostapha
Published: (2026)

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators
by: Li, Jianling, et al.
Published: (2025)

GRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs
by: Dumitran, Adrian-Marius, et al.
Published: (2025)