Cover Image

How Reliable is Language Model Micro-Benchmarking?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yauney, Gregory, Warraich, Shahzaib Saqib, Swayamdipta, Swabha
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrs
by: Ghosh, Sayan, et al.
Published: (2025)

BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
by: Diddee, Harshita, et al.
Published: (2026)

Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information
by: Ethayarajh, Kawin, et al.
Published: (2021)

Stronger Random Baselines for In-Context Learning
by: Yauney, Gregory, et al.
Published: (2024)

Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge
by: Cui, Xinyue, et al.
Published: (2025)

Teaching Models to Understand (but not Generate) High-risk Data
by: Wang, Ryan, et al.
Published: (2025)

Annotating FrameNet via Structure-Conditioned Language Generation
by: Cui, Xinyue, et al.
Published: (2024)

Logits of API-Protected LLMs Leak Proprietary Information
by: Finlayson, Matthew, et al.
Published: (2024)

Compare without Despair: Reliable Preference Evaluation with Generation Separability
by: Ghosh, Sayan, et al.
Published: (2024)

Why Fine-Tuning Encourages Hallucinations and How to Fix It
by: Kaplan, Guy, et al.
Published: (2026)

Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification
by: Liu, Joseph, et al.
Published: (2025)

Disentangling Geometry, Performance, and Training in Language Models
by: Kulkarni, Atharva, et al.
Published: (2026)

Do Large Language Model Benchmarks Test Reliability?
by: Vendrow, Joshua, et al.
Published: (2025)

Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection
by: Kulkarni, Atharva, et al.
Published: (2025)

Improving Language Model Personas via Rationalization with Psychological Scaffolds
by: Joshi, Brihi, et al.
Published: (2025)

Better Language Model Inversion by Compactly Representing Next-Token Distributions
by: Nazir, Murtaza, et al.
Published: (2025)

Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?
by: Khurana, Urja, et al.
Published: (2024)

How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models
by: Frank, Gregory N.
Published: (2026)

Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations
by: He, Keyu, et al.
Published: (2025)

The Reliability Paradox: Exploring How Shortcut Learning Undermines Language Model Calibration
by: Bihani, Geetanjali, et al.
Published: (2024)

TDBench: A Benchmark for Top-Down Image Understanding with Reliability Analysis of Vision-Language Models
by: Hou, Kaiyuan, et al.
Published: (2025)

Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models
by: O'Neill, Charles, et al.
Published: (2024)

On the Reliability of Watermarks for Large Language Models
by: Kirchenbauer, John, et al.
Published: (2023)

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models
by: Zhang, Yang, et al.
Published: (2025)

Reliable Evaluation and Benchmarks for Statement Autoformalization
by: Poiroux, Auguste, et al.
Published: (2024)

Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models
by: Siddiqui, S M Tahmid, et al.
Published: (2026)

How Reliable are Causal Probing Interventions?
by: Canby, Marc, et al.
Published: (2024)

Reliable, Adaptable, and Attributable Language Models with Retrieval
by: Asai, Akari, et al.
Published: (2024)

Context Dependence and Reliability in Autoregressive Language Models
by: Sengupta, Poushali, et al.
Published: (2026)

CEQuest: Benchmarking Large Language Models for Construction Estimation
by: Wu, Yanzhao, et al.
Published: (2025)

On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
by: Gupta, Aarav, et al.
Published: (2026)

Benchmarking Large Language Model Uncertainty for Prompt Optimization
by: Guo, Pei-Fu, et al.
Published: (2024)

Benchmarking Large Language Models for Math Reasoning Tasks
by: Seßler, Kathrin, et al.
Published: (2024)

How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergence
by: Choi, Hyeong Kyu, et al.
Published: (2025)

How Quantization Shapes Bias in Large Language Models
by: Marcuzzi, Federico, et al.
Published: (2025)

How to Train Long-Context Language Models (Effectively)
by: Gao, Tianyu, et al.
Published: (2024)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
by: Chen, Guanjie, et al.
Published: (2024)

Unmasking Hallucinations: A Causal Graph-Attention Perspective on Factual Reliability in Large Language Models
by: kurra, Sailesh kiran, et al.
Published: (2026)

Reliability Under Randomness: An Empirical Analysis of Sparse and Dense Language Models Across Decoding Temperatures
by: Grover, Kabir
Published: (2026)

CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models
by: Wang, Song, et al.
Published: (2024)