Cover Image

MCEval: A Dynamic Framework for Fair Multilingual Cultural Evaluation of LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Shulin, Yang, Linyi, Zhang, Yue
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoning
by: Huang, Shulin, et al.
Published: (2025)

Locking Down the Finetuned LLMs Safety
by: Zhu, Minjun, et al.
Published: (2024)

MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs
by: Zhao, Raoyuan, et al.
Published: (2025)

Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
by: Huang, Shulin, et al.
Published: (2025)

Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs
by: Rystrøm, Jonathan, et al.
Published: (2025)

P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs
by: Zhang, Yidan, et al.
Published: (2024)

How Likely Do LLMs with CoT Mimic Human Reasoning?
by: Bao, Guangsheng, et al.
Published: (2024)

MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs
by: Liu, Zhiwei, et al.
Published: (2025)

MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
by: Mendonça, John, et al.
Published: (2025)

Direct Value Optimization: Improving Chain-of-Thought Reasoning in LLMs with Refined Values
by: Zhang, Hongbo, et al.
Published: (2025)

Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models
by: Zhang, Fan, et al.
Published: (2024)

Personality Alignment of Large Language Models
by: Zhu, Minjun, et al.
Published: (2024)

LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles
by: Huang, Shulin, et al.
Published: (2023)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
by: Vo, Truong, et al.
Published: (2025)

Grounding Multilingual Multimodal LLMs With Cultural Knowledge
by: Nyandwi, Jean de Dieu, et al.
Published: (2025)

Collapsed Language Models Promote Fairness
by: Xu, Jingxuan, et al.
Published: (2024)

MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
by: Zheng, Weihua, et al.
Published: (2025)

An Empirical Analysis of Uncertainty in Large Language Model Evaluations
by: Xie, Qiujie, et al.
Published: (2025)

CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models
by: Hossain, Shehenaz, et al.
Published: (2025)

I Am Aligned, But With Whom? MENA Values Benchmark for Evaluating Cultural Alignment and Multilingual Bias in LLMs
by: Zahraei, Pardis Sadat, et al.
Published: (2025)

Disentangling Language and Culture for Evaluating Multilingual Large Language Models
by: Ying, Jiahao, et al.
Published: (2025)

Cofca: A Step-Wise Counterfactual Multi-hop QA benchmark
by: Wu, Jian, et al.
Published: (2024)

CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs
by: Ye, Yangfan, et al.
Published: (2026)

"Be My Cheese?": Cultural Nuance Benchmarking for Machine Translation in Multilingual LLMs
by: Van Doren, Madison, et al.
Published: (2026)

Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings
by: Liu, Chen Cecilia, et al.
Published: (2023)

A Framework to Assess Multilingual Vulnerabilities of LLMs
by: Tang, Likai, et al.
Published: (2025)

Lessons Without Borders? Evaluating Cultural Alignment of LLMs Using Multilingual Story Moral Generation
by: Wu, Sophie, et al.
Published: (2026)

DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process
by: Zhu, Minjun, et al.
Published: (2025)

Language Fairness in Multilingual Information Retrieval
by: Yang, Eugene, et al.
Published: (2024)

Are LLMs Effective Backbones for Fine-tuning? An Experimental Investigation of Supervised LLMs on Chinese Short Text Matching
by: Liu, Shulin, et al.
Published: (2024)

Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
by: Bao, Guangsheng, et al.
Published: (2023)

Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs
by: Doddapaneni, Sumanth, et al.
Published: (2024)

TASE: Token Awareness and Structured Evaluation for Multilingual Language Models
by: Zhao, Chenzhuo, et al.
Published: (2025)

Multilingual LLMs Are Not Multilingual Thinkers: Evidence from Hindi Analogy Evaluation
by: Gupta, Ashray, et al.
Published: (2025)

FairCoder: Evaluating Social Bias of LLMs in Code Generation
by: Du, Yongkang, et al.
Published: (2025)

LLMs on Trial: Evaluating Judicial Fairness for Large Language Models
by: Hu, Yiran, et al.
Published: (2025)

Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model
by: Pan, Junshu, et al.
Published: (2025)

A Rationale-centric Counterfactual Data Augmentation Method for Cross-Document Event Coreference Resolution
by: Ding, Bowen, et al.
Published: (2024)

Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection
by: He, Jianfei, et al.
Published: (2024)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
by: Singh, Shivalika, et al.
Published: (2024)