Cover Image

LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Etori, Naome A., Lu, Kevin, Karisa, Randu, Kanepajs, Arturs
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Safe Multilingual Frontier AI
by: Kanepajs, Artūrs, et al.
Published: (2024)

Pretraining and Benchmarking Modern Encoders for Latvian
by: Znotins, Arturs
Published: (2026)

RideKE: Leveraging Low-Resource, User-Generated Twitter Content for Sentiment and Emotion Detection in Kenyan Code-Switched Dataset
by: Etori, Naome A., et al.
Published: (2025)

Spanish and LLM Benchmarks: is MMLU Lost in Translation?
by: Plaza, Irene, et al.
Published: (2024)

Mobile-MMLU: A Mobile Intelligence Language Understanding Benchmark
by: Bsharat, Sondos Mahmoud, et al.
Published: (2025)

SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
by: Pramodya, Ashmari, et al.
Published: (2025)

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
by: Wang, Yubo, et al.
Published: (2024)

Performant ASR Models for Medical Entities in Accented Speech
by: Afonja, Tejumade, et al.
Published: (2024)

BnMMLU: Measuring Massive Multitask Language Understanding in Bengali
by: Joy, Saman Sarker, et al.
Published: (2025)

TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
by: Yüksel, Arda, et al.
Published: (2024)

ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic
by: Koto, Fajri, et al.
Published: (2024)

IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledge
by: Abdelaal, Ali, et al.
Published: (2026)

MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark
by: Zhao, Qihao, et al.
Published: (2024)

IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
by: KJ, Sankalp, et al.
Published: (2025)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
by: Singh, Shivalika, et al.
Published: (2024)

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
by: Xuan, Weihao, et al.
Published: (2025)

Are We Done with MMLU?
by: Gema, Aryo Pradipta, et al.
Published: (2024)

Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond
by: Sanni, Mardhiyah, et al.
Published: (2025)

GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greek
by: Zhang, Yang, et al.
Published: (2026)

What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Text
by: Kanepajs, Arturs, et al.
Published: (2025)

DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models
by: Altakrori, Malik H., et al.
Published: (2025)

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
by: Ghahroodi, Omid, et al.
Published: (2024)

1000 African Voices: Advancing inclusive multi-speaker multi-accent speech synthesis
by: Ogun, Sewade, et al.
Published: (2024)

Changing Answer Order Can Decrease MMLU Accuracy
by: Gupta, Vipul, et al.
Published: (2024)

Uhura: A Benchmark for Evaluating Scientific Question Answering and Truthfulness in Low-Resource African Languages
by: Bayes, Edward, et al.
Published: (2024)

MMLU-SR: A Benchmark for Stress-Testing Reasoning Capability of Large Language Models
by: Wang, Wentian, et al.
Published: (2024)

The Veln(ia)s is in the Details: Evaluating LLM Judgment on Latvian and Lithuanian Short Answer Matching
by: Kostiuk, Yevhen, et al.
Published: (2025)

KazMMLU: Evaluating Language Models on Kazakh, Russian, and Regional Knowledge of Kazakhstan
by: Togmanov, Mukhammed, et al.
Published: (2025)

Text Categorization Can Enhance Domain-Agnostic Stopword Extraction
by: Turki, Houcemeddine, et al.
Published: (2024)

MMLU-Pro+: Evaluating Higher-Order Reasoning and Shortcut Learning in LLMs
by: Taghanaki, Saeid Asgari, et al.
Published: (2024)

TEXT2TASTE: A Versatile Egocentric Vision System for Intelligent Reading Assistance Using Large Language Model
by: Mucha, Wiktor, et al.
Published: (2024)

Reactor Mk.1 performances: MMLU, HumanEval and BBH test results
by: Dunham, TJ, et al.
Published: (2024)

EXECUTE: A Multilingual Benchmark for LLM Token Understanding
by: Edman, Lukas, et al.
Published: (2025)

Landscape-Aware Growing: The Power of a Little LAG
by: Karp, Stefani, et al.
Published: (2024)

AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesis
by: Chen, Xuanzhong, et al.
Published: (2025)

LAG: Logic-Augmented Generation from a Cartesian Perspective
by: Xiao, Yilin, et al.
Published: (2025)

Setting Standards in Turkish NLP: TR-MMLU for Large Language Model Evaluation
by: Bayram, M. Ali, et al.
Published: (2024)

QuickScope: Certifying Hard Questions in Dynamic LLM Benchmarks
by: Lundy, Taylor, et al.
Published: (2026)

Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
by: Song, Enxin, et al.
Published: (2025)

AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset
by: Olatunji, Tobi, et al.
Published: (2024)