Cover Image

The Rarity Blind Spot: A Framework for Evaluating Statistical Reasoning in LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Maekawa, Seiji, Iso, Hayate, Bhutani, Nikita
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data
by: Maekawa, Seiji, et al.
Published: (2024)

Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models
by: Maekawa, Seiji, et al.
Published: (2024)

From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization
by: Belem, Catarina G., et al.
Published: (2024)

Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education
by: Iso, Hayate, et al.
Published: (2025)

Less is More for Long Document Summary Evaluation by LLMs
by: Wu, Yunshu, et al.
Published: (2023)

XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updates
by: Zhang, Haopeng, et al.
Published: (2023)

Same Content, Different Representations: A Controlled Study for Table QA
by: Zhang, Yue, et al.
Published: (2025)

Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-$k$
by: Taguchi, Chihiro, et al.
Published: (2025)

AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation
by: Iso, Hayate
Published: (2022)

AmbigNLG: Addressing Task Ambiguity in Instruction for NLG
by: Niwa, Ayana, et al.
Published: (2024)

Noisy Pairing and Partial Supervision for Stylized Opinion Summarization
by: Iso, Hayate, et al.
Published: (2022)

Finding Blind Spots in Evaluator LLMs with Interpretable Checklists
by: Doddapaneni, Sumanth, et al.
Published: (2024)

Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling
by: Maekawa, Seiji, et al.
Published: (2025)

Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviews
by: Shin, Hyungyu, et al.
Published: (2025)

Natural Language Processing for Human Resources: A Survey
by: Otani, Naoki, et al.
Published: (2024)

Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
by: Basmov, Victoria, et al.
Published: (2023)

Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks
by: Zhang, Chuyifei, et al.
Published: (2026)

OmniTQA: A Cost-Aware System for Hybrid Query Processing over Semi-Structured Data
by: Shahbazi, Nima, et al.
Published: (2026)

Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorer
by: Zhang, Jinghan, et al.
Published: (2024)

Unveiling Cultural Blind Spots: Analyzing the Limitations of mLLMs in Procedural Text Comprehension
by: Yari, Amir Hossein, et al.
Published: (2025)

A Theorem-Proving-Based Evaluation of Neural Semantic Parsing
by: Funakura, Hayate, et al.
Published: (2025)

ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities
by: Dong, Wenhan, et al.
Published: (2025)

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions
by: Pezeshkpour, Pouya, et al.
Published: (2024)

Temporal Blind Spots in Large Language Models
by: Wallat, Jonas, et al.
Published: (2024)

Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models
by: Khan, Mohammed Safi Ur Rahman, et al.
Published: (2026)

Align then Train: Efficient Retrieval Adapter Learning
by: Maekawa, Seiji, et al.
Published: (2026)

Do Agents Need to Plan Step-by-Step? Rethinking Planning Horizon in Data-Centric Tool Calling
by: Otani, Naoki, et al.
Published: (2026)

Syntactic Blind Spots: How Misalignment Leads to LLMs Mathematical Errors
by: Williamson, Dane, et al.
Published: (2025)

Linguistic Blind Spots in Clinical Decision Extraction
by: Elgaar, Mohamed, et al.
Published: (2026)

Fluent but Unfeeling: The Emotional Blind Spots of Language Models
by: Shu, Bangzhao, et al.
Published: (2025)

Argument Rarity-based Originality Assessment for AI-Assisted Writing
by: Inoshita, Keito, et al.
Published: (2026)

Linguistic Blind Spots of Large Language Models
by: Cheng, Jiali, et al.
Published: (2025)

Belief in the Machine: Investigating Epistemological Blind Spots of Language Models
by: Suzgun, Mirac, et al.
Published: (2024)

Spot the BlindSpots: Systematic Identification and Quantification of Fine-Grained LLM Biases in Contact Center Summaries
by: Mayilvaghanan, Kawin, et al.
Published: (2025)

Illuminating Blind Spots of Language Models with Targeted Agent-in-the-Loop Synthetic Data
by: Lippmann, Philip, et al.
Published: (2024)

Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP
by: Gautam, Sanjana, et al.
Published: (2024)

Unchecked and Overlooked: Addressing the Checkbox Blind Spot in Large Language Models with CheckboxQA
by: Turski, Michał, et al.
Published: (2025)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
by: Vo, Truong, et al.
Published: (2025)

Decoding Rarity: Large Language Models in the Diagnosis of Rare Diseases
by: Carbonari, Valentina, et al.
Published: (2025)

A Statistical Analysis of LLMs' Self-Evaluation Using Proverbs
by: Sonoda, Ryosuke, et al.
Published: (2024)