Cover Image

MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fabbri, Alexander R., Mares, Diego, Flores, Jorge, Mankikar, Meher, Hernandez, Ernesto, Lee, Dean, Liu, Bing, Xing, Chen
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs
by: Sirdeshmukh, Ved, et al.
Published: (2025)

MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLMs through Multi-hop Hate Speech Explanation
by: Trager, Jackson, et al.
Published: (2025)

Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
by: He, Yun, et al.
Published: (2024)

Challenging Multilingual LLMs: A New Taxonomy and Benchmark for Unraveling Hallucination in Translation
by: Wu, Xinwei, et al.
Published: (2025)

Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems
by: Laban, Philippe, et al.
Published: (2024)

Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning
by: Zhang, Xue, et al.
Published: (2025)

MultiZebraLogic: A Multilingual Logical Reasoning Benchmark
by: Bruun, Sofie Helene, et al.
Published: (2025)

MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs
by: Liu, Zhiwei, et al.
Published: (2025)

P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs
by: Zhang, Yidan, et al.
Published: (2024)

MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
by: Mendonça, John, et al.
Published: (2025)

From Synthetic to Native: Benchmarking Multilingual Intent Classification in Logistics Customer Service
by: He, Haoyu, et al.
Published: (2026)

Evaluating LLMs' Multilingual Capabilities for Bengali: Benchmark Creation and Performance Analysis
by: Bhowmik, Shimanto, et al.
Published: (2025)

CricBench: A Multilingual Benchmark for Evaluating LLMs in Cricket Analytics
by: Agarwal, Parth, et al.
Published: (2025)

Search-Time Data Contamination
by: Han, Ziwen, et al.
Published: (2025)

EnigmaEval: A Benchmark of Long Multimodal Reasoning Challenges
by: Wang, Clinton J., et al.
Published: (2025)

When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
by: Zhao, Weixiang, et al.
Published: (2025)

clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents
by: Beyer, Anne, et al.
Published: (2024)

Shattering the Shortcut: A Topology-Regularized Benchmark for Multi-hop Medical Reasoning in LLMs
by: Zi, Xing, et al.
Published: (2026)

MMATH: A Multilingual Benchmark for Mathematical Reasoning
by: Luo, Wenyang, et al.
Published: (2025)

Ground Truth Generation for Multilingual Historical NLP using LLMs
by: Gladstone, Clovis, et al.
Published: (2025)

SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia
by: Liu, Chaoqun, et al.
Published: (2025)

Multilingual Jailbreak Challenges in Large Language Models
by: Deng, Yue, et al.
Published: (2023)

Unmasking Reasoning Processes: A Process-aware Benchmark for Evaluating Structural Mathematical Reasoning in LLMs
by: Zheng, Xiang, et al.
Published: (2026)

SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs
by: Alam, Firoj, et al.
Published: (2025)

Multilingual European Language Models: Benchmarking Approaches and Challenges
by: Barth, Fabio, et al.
Published: (2025)

Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
by: Fatemi, Bahare, et al.
Published: (2024)

IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages
by: Singh, Harman, et al.
Published: (2024)

XCR-Bench: A Multi-Task Benchmark for Evaluating Cultural Reasoning in LLMs
by: Kabir, Mohsinul, et al.
Published: (2026)

MLissard: Multilingual Long and Simple Sequential Reasoning Benchmarks
by: Bueno, Mirelle, et al.
Published: (2024)

The Riddle of Reflection: Evaluating Reasoning and Self-Awareness in Multilingual LLMs using Indian Riddles
by: M, Abhinav P, et al.
Published: (2025)

Build, Borrow, or Just Fine-Tune? A Political Scientist's Guide to Choosing NLP Models
by: Meher, Shreyas
Published: (2026)

I Am Aligned, But With Whom? MENA Values Benchmark for Evaluating Cultural Alignment and Multilingual Bias in LLMs
by: Zahraei, Pardis Sadat, et al.
Published: (2025)

NativQA: Multilingual Culturally-Aligned Natural Query for LLMs
by: Hasan, Md. Arid, et al.
Published: (2024)

Nondeterministic Polynomial-time Problem Challenge: An Ever-Scaling Reasoning Benchmark for LLMs
by: Yang, Chang, et al.
Published: (2025)

Controlling Language Confusion in Multilingual LLMs
by: Lee, Nahyun, et al.
Published: (2025)

Multilingual LLMs Are Not Multilingual Thinkers: Evidence from Hindi Analogy Evaluation
by: Gupta, Ashray, et al.
Published: (2025)

Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmark
by: Gupta, Himanshu, et al.
Published: (2024)

CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling
by: Wang, Bichen, et al.
Published: (2025)

MultiPragEval: Multilingual Pragmatic Evaluation of Large Language Models
by: Park, Dojun, et al.
Published: (2024)

Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs
by: Doddapaneni, Sumanth, et al.
Published: (2024)