Image de couverture de livre

Watson & Holmes: A Naturalistic Benchmark for Comparing Human and LLM Reasoning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Leelawat, Thatchawin, Griffin, Lewis D
Format:	Preprint
Publié:	2026
Sujets:	Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agents
par: Rombaut, Benjamin, et autres
Publié: (2024)

Temporal Context and Architecture: A Benchmark for Naturalistic EEG Decoding
par: Ergezer, Mehmet
Publié: (2026)

Transcript of GPT-4 playing a rogue AGI in a Matrix Game
par: Griffin, Lewis D, et autres
Publié: (2024)

Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors
par: Hao, Kunkun, et autres
Publié: (2024)

BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
par: Paglieri, Davide, et autres
Publié: (2024)

The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
par: Galatzer-Levy, Isaac R., et autres
Publié: (2024)

ELABORATION: A Comprehensive Benchmark on Human-LLM Competitive Programming
par: Yang, Xinwei, et autres
Publié: (2025)

Human-Level Reasoning: A Comparative Study of Large Language Models on Logical and Abstract Reasoning
par: Moreira, Benjamin Grando
Publié: (2025)

ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimization
par: Tso, Joseph, et autres
Publié: (2026)

ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning
par: Potamitis, Nearchos, et autres
Publié: (2025)

Benchmarking of LLM Detection: Comparing Two Competing Approaches
par: Pröhl, Thorsten, et autres
Publié: (2024)

H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark
par: LeGris, Solim, et autres
Publié: (2024)

HEARTS: Benchmarking LLM Reasoning on Health Time Series
par: Li, Sirui, et autres
Publié: (2026)

Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study
par: Goebel, Kai, et autres
Publié: (2025)

Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insights
par: Parashar, Shubham, et autres
Publié: (2025)

A Representationalist, Functionalist and Naturalistic Conception of Intelligence as a Foundation for AGI
par: Pfister, Rolf
Publié: (2025)

Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
par: Zhao, Tianzhe, et autres
Publié: (2026)

SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
par: Li, Kuan, et autres
Publié: (2026)

LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning
par: Agarwal, Shradha, et autres
Publié: (2026)

An Analysis of Architectural Impact on LLM-based Abstract Visual Reasoning: A Systematic Benchmark on RAVEN-FAIR
par: Urgun, Sinan, et autres
Publié: (2025)

A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences
par: Shen, Jiaxin, et autres
Publié: (2025)

Inducing Personality in LLM-Based Honeypot Agents: Measuring the Effect on Human-Like Agenda Generation
par: Newsham, Lewis, et autres
Publié: (2025)

PrivacyReasoner: Can LLM Emulate a Human-like Privacy Mind?
par: Tu, Yiwen, et autres
Publié: (2026)

MSCoRe: A Benchmark for Multi-Stage Collaborative Reasoning in LLM Agents
par: Lei, Yuzhen, et autres
Publié: (2025)

SPhyR: Spatial-Physical Reasoning Benchmark on Material Distribution
par: Siedler, Philipp D.
Publié: (2025)

Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning
par: Mondorf, Philipp, et autres
Publié: (2024)

LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
par: Kolasani, Sai, et autres
Publié: (2025)

Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimization
par: Lu, Junjie, et autres
Publié: (2025)

ReasoningRec: Bridging Personalized Recommendations and Human-Interpretable Explanations through LLM Reasoning
par: Bismay, Millennium, et autres
Publié: (2024)

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields
par: Liu, Wanhao, et autres
Publié: (2026)

Normative Reasoning in Large Language Models: A Comparative Benchmark from Logical and Modal Perspectives
par: Ozeki, Kentaro, et autres
Publié: (2025)

Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiation
par: Kharlamova, Arina, et autres
Publié: (2025)

Boosting LLM Reasoning via Human-Inspired Reward Shaping
par: Lin, Wenze, et autres
Publié: (2026)

IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluation
par: Tran, Khanh-Tung, et autres
Publié: (2025)

PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning
par: Zhang, Xinyu, et autres
Publié: (2025)

CompareBench: A Benchmark for Visual Comparison Reasoning in Vision-Language Models
par: Cai, Jie, et autres
Publié: (2025)

Formalization of Dialogue in the Decision Support System of Dr. Watson Type
par: Goldberg, Saveli, et autres
Publié: (2024)

AgentDrive: An Open Benchmark Dataset for Agentic AI Reasoning with LLM-Generated Scenarios in Autonomous Systems
par: Ferrag, Mohamed Amine, et autres
Publié: (2026)

Mind the Gap: Evaluating the Representativeness of Quantitative Medical Language Reasoning LLM Benchmarks for African Disease Burdens
par: Mutisya, Fred, et autres
Publié: (2025)

Comparative Analysis and Parametric Tuning of PPO, GRPO, and DAPO for LLM Reasoning Enhancement
par: Lian, Yongsheng
Publié: (2025)