Copertina

Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Xinhe, Huang, Jin, Zhang, Xingjian, Wang, Tianhao, Ma, Jiaqi W.
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning
di: Vaishnav, Mohit, et al.
Pubblicazione: (2026)

Benchmarking Abstract and Reasoning Abilities Through A Theoretical Perspective
di: Ma, Qingchuan, et al.
Pubblicazione: (2025)

Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspective
di: You, Wangjie, et al.
Pubblicazione: (2025)

IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning in Multimodal LLMs
di: Ma, David, et al.
Pubblicazione: (2025)

PerceptionComp: A Video Benchmark for Complex Perception-Centric Reasoning
di: Li, Shaoxuan, et al.
Pubblicazione: (2026)

UGPhysics: A Comprehensive Benchmark for Undergraduate Physics Reasoning with Large Language Models
di: Xu, Xin, et al.
Pubblicazione: (2025)

Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
di: Fatemi, Bahare, et al.
Pubblicazione: (2024)

Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist
di: Zhou, Zihao, et al.
Pubblicazione: (2024)

Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning
di: Zhao, Bingchen, et al.
Pubblicazione: (2024)

Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations
di: Sun, Jiaxing, et al.
Pubblicazione: (2024)

TRAM: Benchmarking Temporal Reasoning for Large Language Models
di: Wang, Yuqing, et al.
Pubblicazione: (2023)

ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
di: Liu, Hongwei, et al.
Pubblicazione: (2025)

MMATH: A Multilingual Benchmark for Mathematical Reasoning
di: Luo, Wenyang, et al.
Pubblicazione: (2025)

FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
di: Tang, Zichen, et al.
Pubblicazione: (2025)

LongReasonArena: A Long Reasoning Benchmark for Large Language Models
di: Ding, Jiayu, et al.
Pubblicazione: (2025)

ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
di: Potamitis, Nearchos, et al.
Pubblicazione: (2025)

StructTest: Benchmarking LLMs' Reasoning through Compositional Structured Outputs
di: Chen, Hailin, et al.
Pubblicazione: (2024)

StressEval: Failure-Driven Dynamic Benchmarking for Knowledge-Intensive Reasoning in Large Language Models
di: Chen, Yongrui, et al.
Pubblicazione: (2026)

THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models
di: Pu, Xiao, et al.
Pubblicazione: (2025)

MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation
di: Li, Xiaoyuan, et al.
Pubblicazione: (2025)

Benchmarking Temporal Reasoning and Alignment Across Chinese Dynasties
di: Wang, Zhenglin, et al.
Pubblicazione: (2025)

SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarios
di: Zhan, Weidong, et al.
Pubblicazione: (2025)

EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models
di: Huang, Junquan, et al.
Pubblicazione: (2025)

AnesSuite: A Comprehensive Benchmark and Dataset Suite for Anesthesiology Reasoning in LLMs
di: Feng, Xiang, et al.
Pubblicazione: (2025)

Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models
di: Yan, Qianqi, et al.
Pubblicazione: (2025)

Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions
di: Hong, Zijin, et al.
Pubblicazione: (2025)

A Reasoning-Focused Legal Retrieval Benchmark
di: Zheng, Lucia, et al.
Pubblicazione: (2025)

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap
di: Srivastava, Saurabh, et al.
Pubblicazione: (2024)

RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models
di: Han, Yunseok, et al.
Pubblicazione: (2026)

Classroom Final Exam: An Instructor-Tested Reasoning Benchmark
di: Gao, Chongyang, et al.
Pubblicazione: (2026)

EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systems
di: Liu, Jingwen, et al.
Pubblicazione: (2025)

ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room
di: Mehandru, Nikita, et al.
Pubblicazione: (2025)

MIND Your Reasoning: A Meta-Cognitive Intuitive-Reflective Network for Dual-Reasoning in Multimodal Stance Detection
di: Wang, Bingbing, et al.
Pubblicazione: (2025)

Unmasking Reasoning Processes: A Process-aware Benchmark for Evaluating Structural Mathematical Reasoning in LLMs
di: Zheng, Xiang, et al.
Pubblicazione: (2026)

MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables
di: Marcuzzo, Matteo, et al.
Pubblicazione: (2025)

UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models
di: Xu, Xin, et al.
Pubblicazione: (2025)

ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
di: Oh, Jungwoo, et al.
Pubblicazione: (2026)

ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models
di: Xue, Boyang, et al.
Pubblicazione: (2025)

SUPERChem: A Multimodal Reasoning Benchmark in Chemistry
di: Zhao, Zehua, et al.
Pubblicazione: (2025)

DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models
di: Zhu, Yakun, et al.
Pubblicazione: (2025)