Copertina

MMLU-SR: A Benchmark for Stress-Testing Reasoning Capability of Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Wentian, Jain, Sarthak, Kantor, Paul, Feldman, Jacob, Gallos, Lazaros, Wang, Hao
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Toward a Metrology for Artificial Intelligence: Hidden-Rule Environments and Reinforcement Learning
di: Mathew, Christo, et al.
Pubblicazione: (2025)

DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models
di: Altakrori, Malik H., et al.
Pubblicazione: (2025)

IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
di: KJ, Sankalp, et al.
Pubblicazione: (2025)

Evaluating Consistency and Reasoning Capabilities of Large Language Models
di: Saxena, Yash, et al.
Pubblicazione: (2024)

MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark
di: Zhao, Qihao, et al.
Pubblicazione: (2024)

Benchmarking the Capabilities of Large Language Models in Transportation System Engineering: Accuracy, Consistency, and Reasoning Behaviors
di: Syed, Usman, et al.
Pubblicazione: (2024)

Exploring and Benchmarking the Planning Capabilities of Large Language Models
di: Bohnet, Bernd, et al.
Pubblicazione: (2024)

Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities
di: Hua, Wenyue, et al.
Pubblicazione: (2024)

Evaluating Interventional Reasoning Capabilities of Large Language Models
di: Kasetty, Tejas, et al.
Pubblicazione: (2024)

Mobile-MMLU: A Mobile Intelligence Language Understanding Benchmark
di: Bsharat, Sondos Mahmoud, et al.
Pubblicazione: (2025)

Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models
di: Jin, Yilun, et al.
Pubblicazione: (2024)

Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization
di: Tang, Zilu, et al.
Pubblicazione: (2025)

Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models
di: Wang, Hengyi, et al.
Pubblicazione: (2024)

Spanish and LLM Benchmarks: is MMLU Lost in Translation?
di: Plaza, Irene, et al.
Pubblicazione: (2024)

Benchmarking Benchmark Leakage in Large Language Models
di: Xu, Ruijie, et al.
Pubblicazione: (2024)

Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
di: Prato, Gabriele, et al.
Pubblicazione: (2025)

ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
di: Oh, Jungwoo, et al.
Pubblicazione: (2026)

UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing
di: He, Yifeng, et al.
Pubblicazione: (2024)

TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models
di: Li, Ce, et al.
Pubblicazione: (2025)

From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
di: Wang, Fei, et al.
Pubblicazione: (2024)

Ko-MuSR: A Multistep Soft Reasoning Benchmark for LLMs Capable of Understanding Korean
di: Park, Chanwoo, et al.
Pubblicazione: (2025)

CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesis
di: Wei, Anjiang, et al.
Pubblicazione: (2025)

Pruning as a Defense: Reducing Memorization in Large Language Models
di: Gupta, Mansi, et al.
Pubblicazione: (2025)

CTBench: A Comprehensive Benchmark for Evaluating Language Model Capabilities in Clinical Trial Design
di: Neehal, Nafis, et al.
Pubblicazione: (2024)

SR-FoT: A Syllogistic-Reasoning Framework of Thought for Large Language Models Tackling Knowledge-based Reasoning Tasks
di: Wan, Wentao, et al.
Pubblicazione: (2025)

SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models
di: Rizvi, Md Imbesat Hassan, et al.
Pubblicazione: (2024)

SSR: Socratic Self-Refine for Large Language Model Reasoning
di: Shi, Haizhou, et al.
Pubblicazione: (2025)

Dissecting Persona-Driven Reasoning in Language Models via Activation Patching
di: Poonia, Ansh, et al.
Pubblicazione: (2025)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
di: Wang, Yiping, et al.
Pubblicazione: (2025)

TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
di: Zhang, Tunyu, et al.
Pubblicazione: (2025)

IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities
di: Wang, Bin, et al.
Pubblicazione: (2024)

Benchmarking the Medical Understanding and Reasoning of Large Language Models in Arabic Healthcare Tasks
di: AlDahoul, Nouar, et al.
Pubblicazione: (2025)

CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models
di: Komanduri, Aneesh, et al.
Pubblicazione: (2025)

PhySense: Principle-Based Physics Reasoning Benchmarking for Large Language Models
di: Xu, Yinggan, et al.
Pubblicazione: (2025)

Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy
di: Dong, Yihong, et al.
Pubblicazione: (2026)

AraLingBench A Human-Annotated Benchmark for Evaluating Arabic Linguistic Capabilities of Large Language Models
di: Zbeeb, Mohammad, et al.
Pubblicazione: (2025)

On Calibration of Large Language Models: From Response To Capability
di: Yang, Sin-Han, et al.
Pubblicazione: (2026)

Are We Done with MMLU?
di: Gema, Aryo Pradipta, et al.
Pubblicazione: (2024)

Generative Evaluation of Complex Reasoning in Large Language Models
di: Lin, Haowei, et al.
Pubblicazione: (2025)

MORSE-500: A Programmatically Controllable Video Benchmark to Stress-Test Multimodal Reasoning
di: Cai, Zikui, et al.
Pubblicazione: (2025)