Image de couverture de livre

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Singh, Shivalika, Romanou, Angelika, Fourrier, Clémentine, Adelani, David I., Ngui, Jian Gang, Vila-Suero, Daniel, Limkonchotiwat, Peerat, Marchisio, Kelly, Leong, Wei Qi, Susanto, Yosephine, Ng, Raymond, Longpre, Shayne, Ko, Wei-Yin, Ruder, Sebastian, Smith, Madeline, Bosselut, Antoine, Oh, Alice, Martins, Andre F. T., Choshen, Leshem, Ippolito, Daphne, Ferrante, Enzo, Fadaee, Marzieh, Ermis, Beyza, Hooker, Sara
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Leaderboard Illusion
par: Singh, Shivalika, et autres
Publié: (2025)

SimMerge: Learning to Select Merge Operators from Similarity Signals
par: Bolton, Oliver, et autres
Publié: (2026)

BURMESE-SAN: Burmese NLP Benchmark for Evaluating Large Language Models
par: Aung, Thura, et autres
Publié: (2026)

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
par: Aakanksha, et autres
Publié: (2024)

SEA-HELM: Southeast Asian Holistic Evaluation of Language Models
par: Susanto, Yosephine, et autres
Publié: (2025)

SEA-SafeguardBench: Evaluating AI Safety in SEA Languages and Cultures
par: Tasawong, Panuthep, et autres
Publié: (2025)

SEA-Guard: Culturally Grounded Multilingual Safeguard for Southeast Asia
par: Tasawong, Panuthep, et autres
Publié: (2026)

SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?
par: Ponwitayarat, Wuttikorn, et autres
Publié: (2025)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
par: Aakanksha, et autres
Publié: (2024)

From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models
par: Pozzobon, Luiza, et autres
Publié: (2024)

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives
par: Shimabucoro, Luísa, et autres
Publié: (2024)

The Multilingual Divide and Its Impact on Global AI Safety
par: Peppin, Aidan, et autres
Publié: (2025)

Future and AI-Ready Data Strategies: Response to DOC RFI on AI and Open Government Data Assets
par: Oderinwale, Hamidah, et autres
Publié: (2024)

Kalahi: A handcrafted, grassroots cultural LLM evaluation suite for Filipino
par: Montalan, Jann Railey, et autres
Publié: (2024)

The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It
par: Yong, Zheng-Xin, et autres
Publié: (2025)

Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress
par: Odumakinde, Ayomide, et autres
Publié: (2024)

PERK: Long-Context Reasoning as Parameter-Efficient Test-Time Learning
par: Chen, Zeming, et autres
Publié: (2025)

Space Decomposition for Sentence Embedding
par: Ponwitayarat, Wuttikorn, et autres
Publié: (2024)

Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai
par: Pengpun, Parinthapat, et autres
Publié: (2024)

Can Gradient Descent Simulate Prompting?
par: Zhang, Eric, et autres
Publié: (2025)

A Hitchhiker's Guide to Scaling Law Estimation
par: Choshen, Leshem, et autres
Publié: (2024)

Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion
par: Zaman, Kerem, et autres
Publié: (2023)

How Does Quantization Affect Multilingual LLMs?
par: Marchisio, Kelly, et autres
Publié: (2024)

A Post-trainer's Guide to Multilingual Training Data: Uncovering Cross-lingual Transfer Dynamics
par: Shimabucoro, Luisa, et autres
Publié: (2025)

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
par: Üstün, Ahmet, et autres
Publié: (2024)

Understanding and Mitigating Language Confusion in LLMs
par: Marchisio, Kelly, et autres
Publié: (2024)

When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQA
par: Tuchinda, Pume, et autres
Publié: (2025)

The ShareLM Collection and Plugin: Contributing Human-Model Chats for the Benefit of the Community
par: Don-Yehiya, Shachar, et autres
Publié: (2024)

Naturally Occurring Feedback is Common, Extractable and Useful
par: Don-Yehiya, Shachar, et autres
Publié: (2024)

A Systematic Review of NeurIPS Dataset Management Practices
par: Wu, Yiwei, et autres
Publié: (2024)

Instructions Shape Production of Language, not Processing
par: Waldis, Andreas, et autres
Publié: (2026)

ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization
par: Yadav, Prateek, et autres
Publié: (2023)

AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic
par: Robinson, Nathaniel R., et autres
Publié: (2024)

AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Research
par: Simmons-Edler, Riley, et autres
Publié: (2024)

CAVE: Detecting and Explaining Commonsense Anomalies in Visual Environments
par: Bhagwatkar, Rishika, et autres
Publié: (2025)

Decom-Renorm-Merge: Model Merging on the Right Space Improves Multitasking
par: Chaichana, Yuatyong, et autres
Publié: (2025)

Distilling Multilingual Vision-Language Models: When Smaller Models Stay Multilingual
par: Sriratanawilai, Sukrit, et autres
Publié: (2025)

Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
par: Din, Alexander Yom, et autres
Publié: (2023)

Mediocrity is the key for LLM as a Judge Anchor Selection
par: Don-Yehiya, Shachar, et autres
Publié: (2026)

Pretraining Language Models for Diachronic Linguistic Change Discovery
par: Fittschen, Elisabeth, et autres
Publié: (2025)