Cover Image

The Dog the Cat Chased Stumped the Model: Measuring When Language Models Abandon Structure for Shortcuts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Madhusudan, Sangmitra, Chen, Kaige, Emami, Ali
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions
by: Morabito, Robert, et al.
Published: (2024)

NYT-Connections: A Deceptively Simple Text Classification Task that Stumps System-1 Thinkers
by: Lopez, Angel Yahir Loredo, et al.
Published: (2024)

Which Words Matter Most in Zero-Shot Prompts?
by: Sadr, Nikta Gohari, et al.
Published: (2025)

Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models
by: Kumar, Abhishek, et al.
Published: (2024)

Fine-Tuned LLMs are "Time Capsules" for Tracking Societal Bias Through Books
by: Madhusudan, Sangmitra, et al.
Published: (2025)

MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Models
by: Yunusov, Sarfaroz, et al.
Published: (2024)

Metamorphic Testing for Fairness Evaluation in Large Language Models: Identifying Intersectional Bias in LLaMA and GPT
by: Reddy, Harishwar, et al.
Published: (2025)

Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models
by: Mikula, Lukáš, et al.
Published: (2023)

Break the Chain: Large Language Models Can be Shortcut Reasoners
by: Ding, Mengru, et al.
Published: (2024)

Common to Whom? Regional Cultural Commonsense and LLM Bias in India
by: Madhusudan, Sangmitra, et al.
Published: (2026)

TALE: A Tool-Augmented Framework for Reference-Free Evaluation of Large Language Models
by: Badshah, Sher, et al.
Published: (2025)

Mitigating Shortcut Reasoning in Language Models: A Gradient-Aware Training Approach
by: Cao, Hongyu, et al.
Published: (2026)

Learning Shortcuts: On the Misleading Promise of NLU in Language Models
by: Bihani, Geetanjali, et al.
Published: (2024)

Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
by: Lehmann, Hans Hergen, et al.
Published: (2025)

Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
by: Kumar, Abhishek, et al.
Published: (2024)

Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition
by: McDonald, Tyler, et al.
Published: (2025)

Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models
by: Sakib, Fardin Ahsan, et al.
Published: (2025)

Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models
by: Huang, Chengkai, et al.
Published: (2024)

Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding
by: Chi, Ziheng, et al.
Published: (2025)

The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
by: Kim, Dueun, et al.
Published: (2026)

Efficient Fairness Testing in Large Language Models: Prioritizing Metamorphic Relations for Bias Detection
by: Giramata, Suavis, et al.
Published: (2025)

The Reliability Paradox: Exploring How Shortcut Learning Undermines Language Model Calibration
by: Bihani, Geetanjali, et al.
Published: (2024)

Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?
by: Li, Bangzheng, et al.
Published: (2023)

When Do Tools and Planning Help Large Language Models Think? A Cost- and Latency-Aware Benchmark
by: Ghoshal, Subha, et al.
Published: (2026)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
by: Chen, Yanjun, et al.
Published: (2024)

When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour
by: Ranaldi, Leonardo, et al.
Published: (2023)

SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping
by: Lu, Yu-Chen, et al.
Published: (2025)

When Quantization Affects Confidence of Large Language Models?
by: Proskurina, Irina, et al.
Published: (2024)

CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models
by: Kocak, Aysenur, et al.
Published: (2025)

When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors
by: Emmons, Scott, et al.
Published: (2025)

Rectifying Demonstration Shortcut in In-Context Learning
by: Jang, Joonwon, et al.
Published: (2024)

Cognitive Decision Routing in Large Language Models: When to Think Fast, When to Think Slow
by: Du, Y., et al.
Published: (2025)

Chasing Random: Instruction Selection Strategies Fail to Generalize
by: Diddee, Harshita, et al.
Published: (2024)

Measuring Form and Function in Language Models
by: Martínez, Héctor Javier Vázquez, et al.
Published: (2026)

Large Language Models Often Know When They Are Being Evaluated
by: Needham, Joe, et al.
Published: (2025)

When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models
by: Wu, Huyu, et al.
Published: (2025)

Adaptive Large Language Models By Layerwise Attention Shortcuts
by: Verma, Prateek, et al.
Published: (2024)

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics
by: Somasekharan, Nithin, et al.
Published: (2025)

Shortcut Learning in In-Context Learning: A Survey
by: Song, Rui, et al.
Published: (2024)

On the Shortcut Learning in Multilingual Neural Machine Translation
by: Wang, Wenxuan, et al.
Published: (2024)