Cover Image

BnMMLU: Measuring Massive Multitask Language Understanding in Bengali

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Joy, Saman Sarker, Shatabda, Swakkhar
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
by: Yüksel, Arda, et al.
Published: (2024)

ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic
by: Koto, Fajri, et al.
Published: (2024)

SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
by: Pramodya, Ashmari, et al.
Published: (2025)

KMMLU: Measuring Massive Multitask Language Understanding in Korean
by: Son, Guijin, et al.
Published: (2024)

Eyes on the Image: Gaze Supervised Multimodal Learning for Chest X-ray Diagnosis and Report Generation
by: Riju, Tanjim Islam, et al.
Published: (2025)

Understanding QA generation: Extracting Parametric and Contextual Knowledge with CQA for Low Resource Bangla Language
by: Azmary, Umme Abira, et al.
Published: (2026)

DPCSpell: A Transformer-based Detector-Purificator-Corrector Framework for Spelling Error Correction of Bangla and Resource Scarce Indic Languages
by: Bijoy, Mehedi Hasan, et al.
Published: (2022)

Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
by: Sayeedi, Nurul Labib, et al.
Published: (2026)

Measuring Massive Multitask Chinese Understanding
by: Zeng, Hui
Published: (2023)

GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greek
by: Zhang, Yang, et al.
Published: (2026)

BnSentMix: A Diverse Bengali-English Code-Mixed Dataset for Sentiment Analysis
by: Alam, Sadia, et al.
Published: (2024)

BanglishRev: A Large-Scale Bangla-English and Code-mixed Dataset of Product Reviews in E-Commerce
by: Shamael, Mohammad Nazmush, et al.
Published: (2024)

An Enhanced Text Compression Approach Using Transformer-based Language Models
by: Rahman, Chowdhury Mofizur, et al.
Published: (2024)

Mobile-MMLU: A Mobile Intelligence Language Understanding Benchmark
by: Bsharat, Sondos Mahmoud, et al.
Published: (2025)

Can LLMs Solve My Grandma's Riddle? Evaluating Multilingual Large Language Models on Reasoning Traditional Bangla Tricky Riddles
by: Sayeedi, Nurul Labib, et al.
Published: (2025)

Gazetteer-Enhanced Bangla Named Entity Recognition with BanglaBERT Semantic Embeddings K-Means-Infused CRF Model
by: Farhan, Niloy, et al.
Published: (2024)

FeatPCA: A feature subspace based principal component analysis technique for enhancing clustering of single-cell RNA-seq data
by: Islam, Md Romizul, et al.
Published: (2025)

Explainable Multimodal Sentiment Analysis on Bengali Memes
by: Elahi, Kazi Toufique, et al.
Published: (2023)

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
by: Wang, Yubo, et al.
Published: (2024)

LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama
by: Etori, Naome A., et al.
Published: (2025)

MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning
by: Sobhani, Mahbub E, et al.
Published: (2025)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
by: Singh, Shivalika, et al.
Published: (2024)

IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding
by: KJ, Sankalp, et al.
Published: (2025)

MultiMed: Massively Multimodal and Multitask Medical Understanding
by: Mo, Shentong, et al.
Published: (2024)

Are We Done with MMLU?
by: Gema, Aryo Pradipta, et al.
Published: (2024)

MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark
by: Zhao, Qihao, et al.
Published: (2024)

Measuring Hong Kong Massive Multi-Task Language Understanding
by: Cao, Chuxue, et al.
Published: (2025)

KazMMLU: Evaluating Language Models on Kazakh, Russian, and Regional Knowledge of Kazakhstan
by: Togmanov, Mukhammed, et al.
Published: (2025)

Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domains
by: Sayeedi, Md. Faiyaz Abdullah, et al.
Published: (2025)

FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluation
by: Tanzer, Garrett
Published: (2024)

BengaliMoralBench: A Benchmark for Auditing Moral Reasoning in Large Language Models within Bengali Language and Culture
by: Ridoy, Shahriyar Zaman, et al.
Published: (2025)

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
by: Xuan, Weihao, et al.
Published: (2025)

Changing Answer Order Can Decrease MMLU Accuracy
by: Gupta, Vipul, et al.
Published: (2024)

A systematic review of metaheuristics-based and machine learning-driven intrusion detection systems in IoT
by: Ahsan, Mohammad Shamim, et al.
Published: (2025)

VisText-Mosquito: A Unified Multimodal Dataset for Visual Detection, Segmentation, and Textual Explanation on Mosquito Breeding Sites
by: Islam, Md. Adnanul, et al.
Published: (2025)

M3TQA: Massively Multilingual Multitask Table Question Answering
by: Shu, Daixin, et al.
Published: (2025)

Spanish and LLM Benchmarks: is MMLU Lost in Translation?
by: Plaza, Irene, et al.
Published: (2024)

DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models
by: Altakrori, Malik H., et al.
Published: (2025)

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
by: Ghahroodi, Omid, et al.
Published: (2024)

IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledge
by: Abdelaal, Ali, et al.
Published: (2026)