Copertina

Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Mousavi, Seyed Mahed, Cecchinato, Edoardo, Hornikova, Lucia, Riccardi, Giuseppe
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

What Does Loss Optimization Actually Teach, If Anything? Knowledge Dynamics in Continual Pre-training of LLMs
di: Mousavi, Seyed Mahed, et al.
Pubblicazione: (2026)

LLMs as Repositories of Factual Knowledge: Limitations and Solutions
di: Mousavi, Seyed Mahed, et al.
Pubblicazione: (2025)

DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMs
di: Mousavi, Seyed Mahed, et al.
Pubblicazione: (2024)

[De|Re]constructing VLMs' Reasoning in Counting
di: Alghisi, Simone, et al.
Pubblicazione: (2025)

Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialogue
di: Alghisi, Simone, et al.
Pubblicazione: (2024)

Are LLMs Robust for Spoken Dialogues?
di: Mousavi, Seyed Mahed, et al.
Pubblicazione: (2024)

CIVET: Systematic Evaluation of Understanding in VLMs
di: Rizzoli, Massimo, et al.
Pubblicazione: (2025)

Getting to the Point: Pointing Improves LVLMs at Counting
di: Alghisi, Simone, et al.
Pubblicazione: (2026)

V-DyKnow: A Dynamic Benchmark for Time-Sensitive Knowledge in Vision Language Models
di: Mousavi, Seyed Mahed, et al.
Pubblicazione: (2026)

MATEO: A Multimodal Benchmark for Temporal Reasoning and Planning in LVLMs
di: Roccabruna, Gabriel, et al.
Pubblicazione: (2026)

What Are They Doing? Joint Audio-Speech Co-Reasoning
di: Wang, Yingzhi, et al.
Pubblicazione: (2024)

The Reasoning Error About Reasoning: Why Different Types of Reasoning Require Different Representational Structures
di: Wu, Yiling
Pubblicazione: (2026)

What Are They Talking About? A Benchmark of Knowledge-Grounded Discussion Summarization
di: Zhou, Weixiao, et al.
Pubblicazione: (2025)

Virtual Garbage Collector (VGC): A Zone-Based Garbage Collection Architecture for Python's Parallel Runtime
di: M, Abdulla
Pubblicazione: (2025)

What Do Speech Foundation Models Not Learn About Speech?
di: Waheed, Abdul, et al.
Pubblicazione: (2024)

LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs
di: Davoodi, Arash Gholami, et al.
Pubblicazione: (2024)

Thinking Out Loud: Do Reasoning Models Know When They're Right?
di: Zeng, Qingcheng, et al.
Pubblicazione: (2025)

What Are They Filtering Out? An Experimental Benchmark of Filtering Strategies for Harm Reduction in Pretraining Datasets
di: Stranisci, Marco Antonio, et al.
Pubblicazione: (2025)

What Do AI Agents Talk About? Discourse and Architectural Constraints in the First AI-Only Social Network
di: Dube, Taksch, et al.
Pubblicazione: (2026)

Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?
di: Kang, Deokhyung, et al.
Pubblicazione: (2025)

Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
di: Wu, Mingqi, et al.
Pubblicazione: (2025)

Classifying Unreliable Narrators with Large Language Models
di: Brei, Anneliese, et al.
Pubblicazione: (2025)

What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practices
di: Noels, Sander, et al.
Pubblicazione: (2025)

What Do Self-Supervised Speech Models Know About Words?
di: Pasad, Ankita, et al.
Pubblicazione: (2023)

Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning
di: Sok, Jaewon, et al.
Pubblicazione: (2026)

HearSay Benchmark: Do Audio LLMs Leak What They Hear?
di: Wang, Jin, et al.
Pubblicazione: (2026)

Will LLMs Replace the Encoder-Only Models in Temporal Relation Classification?
di: Roccabruna, Gabriel, et al.
Pubblicazione: (2024)

PARSE: An Open-Domain Reasoning Question Answering Benchmark for Persian
di: Mozafari, Jamshid, et al.
Pubblicazione: (2026)

BeDiscovER: The Benchmark of Discourse Understanding in the Era of Reasoning Language Models
di: Li, Chuyuan, et al.
Pubblicazione: (2025)

What the HellaSwag? On the Validity of Common-Sense Reasoning Benchmarks
di: Chizhov, Pavel, et al.
Pubblicazione: (2025)

A Primer in Post-Training Reasoning Data: What We Know About How It Works
di: Li, Yaoming, et al.
Pubblicazione: (2026)

Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
di: Damirchi, Hamed, et al.
Pubblicazione: (2026)

Do Reasoning LLMs Refuse What They Infer in Long Contexts?
di: Fu, Yu, et al.
Pubblicazione: (2026)

Reasoning Models Will Sometimes Lie About Their Reasoning
di: Walden, William, et al.
Pubblicazione: (2026)

What Do LLMs Know About Alzheimer's Disease? Multi-loss Fine-Tuning and Probing for AD Detection
di: Jiang, Lei, et al.
Pubblicazione: (2026)

Towards Efficient Patient Recruitment for Clinical Trials: Application of a Prompt-Based Learning Model
di: Rahmanian, Mojdeh, et al.
Pubblicazione: (2024)

What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis
di: Rao, Delip, et al.
Pubblicazione: (2026)

WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning
di: Mundada, Gagan, et al.
Pubblicazione: (2025)

Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation
di: Li, Zhuohang, et al.
Pubblicazione: (2024)

It Is Not About What You Say, It Is About How You Say It: A Surprisingly Simple Approach for Improving Reading Comprehension
di: Shaier, Sagi, et al.
Pubblicazione: (2024)