Image de couverture de livre

MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Weiqi, Song, Yangqiu
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
par: Wang, Zhaowei, et autres
Publié: (2023)

ConKE: Conceptualization-Augmented Knowledge Editing in Large Language Models for Commonsense Reasoning
par: Zhang, Liyu, et autres
Publié: (2024)

IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce
par: Ding, Wenxuan, et autres
Publié: (2024)

Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?
par: Liu, Jiayu, et autres
Publié: (2025)

arXiv2Table: Toward Realistic Benchmarking and Evaluation for LLM-Based Literature-Review Table Generation
par: Wang, Weiqi, et autres
Publié: (2025)

ActPlan-1K: Benchmarking the Procedural Planning Ability of Visual Language Models in Household Activities
par: Su, Ying, et autres
Publié: (2024)

Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization
par: He, Mutian, et autres
Publié: (2022)

GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory
par: Fan, Wei, et autres
Publié: (2024)

SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding
par: Yang, Yuqi, et autres
Publié: (2025)

Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Framework
par: Deng, Zheye, et autres
Publié: (2025)

Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models
par: Lin, Zizheng, et autres
Publié: (2024)

Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction
par: Han, Kaiqiao, et autres
Publié: (2024)

From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery
par: Zheng, Tianshi, et autres
Publié: (2025)

McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models
par: Lan, Tian, et autres
Publié: (2025)

CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge
par: Zheng, Tianshi, et autres
Publié: (2024)

CKBP v2: Better Annotation and Reasoning for Commonsense Knowledge Base Population
par: Fang, Tianqing, et autres
Publié: (2023)

SeedBench: A Multi-task Benchmark for Evaluating Large Language Models in Seed Science
par: Ying, Jie, et autres
Publié: (2025)

CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning
par: Wang, Weiqi, et autres
Publié: (2024)

KnowShiftQA: How Robust are RAG Systems when Textbook Knowledge Shifts in K-12 Education?
par: Zheng, Tianshi, et autres
Publié: (2024)

PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models
par: Li, Haoran, et autres
Publié: (2023)

MARS: Enabling Autoregressive Models Multi-Token Generation
par: Jin, Ziqi, et autres
Publié: (2026)

ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations
par: Chan, Chunkit, et autres
Publié: (2023)

Towards Multi-Agent Reasoning Systems for Collaborative Expertise Delegation: An Exploratory Design Study
par: Xu, Baixuan, et autres
Publié: (2025)

MASLegalBench: Benchmarking Multi-Agent Systems in Deductive Legal Reasoning
par: Jing, Huihao, et autres
Publié: (2025)

EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association
par: Wang, Weiqi, et autres
Publié: (2025)

RESPONSE: Benchmarking the Ability of Language Models to Undertake Commonsense Reasoning in Crisis Situation
par: Diallo, Aissatou, et autres
Publié: (2025)

Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
par: Wang, Qineng, et autres
Publié: (2024)

Evaluating the Ability of Large Language Models to Reason about Cardinal Directions
par: Cohn, Anthony G, et autres
Publié: (2024)

Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models
par: Chen, Zhipeng, et autres
Publié: (2024)

LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
par: Gui, Jiayi, et autres
Publié: (2024)

MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation
par: Yang, Chenghao, et autres
Publié: (2025)

L-MARS: Legal Multi-Agent Workflow with Orchestrated Reasoning and Agentic Search
par: Wang, Ziqi, et autres
Publié: (2025)

ConstraintChecker: A Plugin for Large Language Models to Reason on Commonsense Knowledge Bases
par: Do, Quyet V., et autres
Publié: (2024)

Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
par: Hagendorff, Thilo, et autres
Publié: (2025)

Disentangling Memory and Reasoning Ability in Large Language Models
par: Jin, Mingyu, et autres
Publié: (2024)

Optimizing Language Model's Reasoning Abilities with Weak Supervision
par: Tong, Yongqi, et autres
Publié: (2024)

Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models
par: Patel, Nisarg, et autres
Publié: (2024)

MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark
par: Wang, Dingdong, et autres
Publié: (2025)

FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
par: Luo, Junyu, et autres
Publié: (2025)

M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models
par: Kwan, Wai-Chung, et autres
Publié: (2023)