Image de couverture de livre

QuickScope: Certifying Hard Questions in Dynamic LLM Benchmarks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lundy, Taylor, Raman, Narun K., Leyton-Brown, Kevin
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Reasoning Models are Test Exploiters: Rethinking Multiple-Choice
par: Raman, Narun, et autres
Publié: (2025)

STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models
par: Raman, Narun, et autres
Publié: (2025)

STEER: Assessing the Economic Rationality of Large Language Models
par: Raman, Narun, et autres
Publié: (2024)

NFTs as a Data-Rich Test Bed: Conspicuous Consumption and its Determinants
par: Lundy, Taylor, et autres
Publié: (2025)

Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models
par: Leyton-Brown, Kevin, et autres
Publié: (2024)

Generating Hard-Negative Out-of-Scope Data with ChatGPT for Intent Classification
par: Li, Zhijian, et autres
Publié: (2024)

Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?
par: Balepur, Nishant, et autres
Publié: (2024)

ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questions
par: Peng, Zhiyuan, et autres
Publié: (2024)

Generating Benchmarks for Factuality Evaluation of Language Models
par: Muhlgay, Dor, et autres
Publié: (2023)

Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks
par: Yugeswardeenoo, Dharunish, et autres
Publié: (2024)

Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
par: Kiruluta, Andrew, et autres
Publié: (2025)

Beyond Easy Wins: A Text Hardness-Aware Benchmark for LLM-generated Text Detection
par: Ayoobi, Navid, et autres
Publié: (2025)

Certified Mitigation of Worst-Case LLM Copyright Infringement
par: Zhang, Jingyu, et autres
Publié: (2025)

FReM: A Flexible Reasoning Mechanism for Balancing Quick and Slow Thinking in Long-Context Question Answering
par: Zhao, Zhengyi, et autres
Publié: (2025)

HalluHard: A Hard Multi-Turn Hallucination Benchmark
par: Fan, Dongyang, et autres
Publié: (2026)

FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation
par: Kiruluta, Andrew, et autres
Publié: (2025)

Beyond Self Attention: A Subquadratic Fourier Wavelet Transformer with Multi Modal Fusion
par: Kiruluta, Andrew, et autres
Publié: (2021)

DEEPAMBIGQA: Ambiguous Multi-hop Questions for Benchmarking LLM Answer Completeness
par: Ji, Jiabao, et autres
Publié: (2025)

MEQA: A Meta-Evaluation Framework for Question & Answer LLM Benchmarks
par: Veuthey, Jaime Raldua, et autres
Publié: (2025)

Give me Some Hard Questions: Synthetic Data Generation for Clinical QA
par: Bai, Fan, et autres
Publié: (2024)

Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmark
par: Xu, Zhikun, et autres
Publié: (2024)

The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights
par: Zhu, Wenhao, et autres
Publié: (2024)

LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama
par: Etori, Naome A., et autres
Publié: (2025)

FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering
par: Lee, Gyubok, et autres
Publié: (2025)

IMB: An Italian Medical Benchmark for Question Answering
par: Romano, Antonio, et autres
Publié: (2025)

Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
par: Bergner, Benjamin, et autres
Publié: (2024)

Platypus: Quick, Cheap, and Powerful Refinement of LLMs
par: Lee, Ariel N., et autres
Publié: (2023)

Learned, Lagged, LLM-splained: LLM Responses to End User Security Questions
par: Prakash, Vijay, et autres
Publié: (2024)

PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answering
par: Meem, Jannat Ara, et autres
Publié: (2024)

HELEA: Hard-Negative Benchmark and LLM-based Reranking for Robust Entity Alignment
par: Jang, Yoonjin, et autres
Publié: (2026)

MALIBU Benchmark: Multi-Agent LLM Implicit Bias Uncovered
par: Mirza, Imran, et autres
Publié: (2025)

A Dataset and Benchmark for Consumer Healthcare Question Summarization
par: Basu, Abhishek, et autres
Publié: (2025)

Argument-Based Comparative Question Answering Evaluation Benchmark
par: Nikishina, Irina, et autres
Publié: (2025)

MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering
par: Zhang, Xuanliang, et autres
Publié: (2025)

MHQA: A Diverse, Knowledge Intensive Mental Health Question Answering Challenge for Language Models
par: Racha, Suraj, et autres
Publié: (2025)

TopoBench: Benchmarking LLMs on Hard Topological Reasoning
par: Maniparambil, Mayug, et autres
Publié: (2026)

Question: How do Large Language Models perform on the Question Answering tasks? Answer:
par: Fischer, Kevin, et autres
Publié: (2024)

AAVENUE: Detecting LLM Biases on NLU Tasks in AAVE via a Novel Benchmark
par: Gupta, Abhay, et autres
Publié: (2024)

HardTests: Synthesizing High-Quality Test Cases for LLM Coding
par: He, Zhongmou, et autres
Publié: (2025)

Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation
par: Wang, Siyuan, et autres
Publié: (2024)