Cover Image

The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Deveci, İbrahim Ethem, Ataman, Duygu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluation
by: Sälevä, Jonne, et al.
Published: (2025)

Generalization Measures for Zero-Shot Cross-Lingual Transfer
by: Bassi, Saksham, et al.
Published: (2024)

Open foundation models for Azerbaijani language
by: Isbarov, Jafar, et al.
Published: (2024)

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
by: Zhao, Weilin, et al.
Published: (2024)

Evaluating Morphological Compositional Generalization in Large Language Models
by: Ismayilzada, Mete, et al.
Published: (2024)

D-NLP at SemEval-2024 Task 2: Evaluating Clinical Inference Capabilities of Large Language Models
by: Altinok, Duygu
Published: (2024)

Introducing TrGLUE and SentiTurca: A Comprehensive Benchmark for Turkish General Language Understanding and Sentiment Analysis
by: Altinok, Duygu
Published: (2025)

BeDiscovER: The Benchmark of Discourse Understanding in the Era of Reasoning Language Models
by: Li, Chuyuan, et al.
Published: (2025)

Optimal Turkish Subword Strategies at Scale: Systematic Evaluation of Data, Vocabulary, Morphology Interplay
by: Altinok, Duygu
Published: (2026)

Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks
by: Cao, Yixin, et al.
Published: (2025)

Enhancing Human-Like Responses in Large Language Models
by: Çalık, Ethem Yağız, et al.
Published: (2025)

A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
by: Yan, Yibo, et al.
Published: (2024)

CRG Score: A Distribution-Aware Clinical Metric for Radiology Report Generation
by: Hamamci, Ibrahim Ethem, et al.
Published: (2025)

SEAL: Can Saturated Benchmarks Be Revived by LLM-as-a-Meta-Judge?
by: Chen, Jiamin, et al.
Published: (2026)

TUMLU: A Unified and Native Language Understanding Benchmark for Turkic Languages
by: Isbarov, Jafar, et al.
Published: (2025)

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap
by: Srivastava, Saurabh, et al.
Published: (2024)

SyncThink: A Training-Free Strategy to Align Inference Termination with Reasoning Saturation
by: Li, Gengyang, et al.
Published: (2026)

Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
by: Fatemi, Bahare, et al.
Published: (2024)

Whispering Context: Distilling Syntax and Semantics for Long Speech Transcripts
by: Altinok, Duygu
Published: (2025)

Saturation-Driven Dataset Generation for LLM Mathematical Reasoning in the TPTP Ecosystem
by: Quesnel, Valentin, et al.
Published: (2025)

MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation
by: Li, Xiaoyuan, et al.
Published: (2025)

FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
by: Luo, Junyu, et al.
Published: (2025)

EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models
by: Huang, Junquan, et al.
Published: (2025)

Benchmarks Saturate When The Model Gets Smarter Than The Judge
by: Ballon, Marthe, et al.
Published: (2026)

RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation
by: Xu, Xinnuo, et al.
Published: (2025)

Your Reasoning Benchmark May Not Test Reasoning: Revealing Perception Bottleneck in Abstract Reasoning Benchmarks
by: Wang, Xinhe, et al.
Published: (2025)

ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
by: Oh, Jungwoo, et al.
Published: (2026)

SocialMaze: A Benchmark for Evaluating Social Reasoning in Large Language Models
by: Xu, Zixiang, et al.
Published: (2025)

A Japanese Benchmark for Evaluating Social Bias in Reasoning Based on Attribution Theory
by: Shiotani, Taihei, et al.
Published: (2026)

GroundCocoa: A Benchmark for Evaluating Compositional & Conditional Reasoning in Language Models
by: Kohli, Harsh, et al.
Published: (2024)

Can Deception Detection Go Deeper? Dataset, Evaluation, and Benchmark for Deception Reasoning
by: Chen, Kang, et al.
Published: (2024)

RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Models
by: Wang, Yuqing, et al.
Published: (2024)

CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Models
by: Chun, Jon, et al.
Published: (2026)

ScholarBench: A Bilingual Benchmark for Abstraction, Comprehension, and Reasoning Evaluation in Academic Contexts
by: Noh, Dongwon, et al.
Published: (2025)

TRAVELER: A Benchmark for Evaluating Temporal Reasoning across Vague, Implicit and Explicit References
by: Kenneweg, Svenja, et al.
Published: (2025)

BaZi-Based Character Simulation Benchmark: Evaluating AI on Temporal and Persona Reasoning
by: Zheng, Siyuan, et al.
Published: (2025)

BaziQA-Benchmark: Evaluating Symbolic and Temporally Compositional Reasoning in Large Language Models
by: Chen, Jiangxi, et al.
Published: (2026)

Boosting CTC-Based ASR Using LLM-Based Intermediate Loss Regularization
by: Altinok, Duygu
Published: (2025)

Mind the Gap: Entity-Preserved Context-Aware ASR Structured Transcriptions
by: Altinok, Duygu
Published: (2025)

Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning
by: Kim, Minwu, et al.
Published: (2026)