Cover Image

mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Raihan, Nishat, Anastasopoulos, Antonios, Zampieri, Marcos
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla
by: Raihan, Nishat, et al.
Published: (2025)

EmoMix-3L: A Code-Mixed Dataset for Bangla-English-Hindi Emotion Detection
by: Raihan, Nishat, et al.
Published: (2024)

TigerLLM - A Family of Bangla Large Language Models
by: Raihan, Nishat, et al.
Published: (2025)

Temporal Text Classification with Large Language Models
by: Raihan, Nishat, et al.
Published: (2026)

A Taxonomy of Programming Languages for Code Generation
by: Raihan, Nishat, et al.
Published: (2026)

MojoBench: Language Modeling and Benchmarks for Mojo
by: Raihan, Nishat, et al.
Published: (2024)

Large Language Models for Mental Health: A Multilingual Evaluation
by: Raihan, Nishat, et al.
Published: (2026)

Code LLMs: A Taxonomy-based Survey
by: Raihan, Nishat, et al.
Published: (2024)

An Efficient Approach for Studying Cross-Lingual Transfer in Multilingual Language Models
by: Faisal, Fahim, et al.
Published: (2024)

CSEPrompts: A Benchmark of Introductory Computer Science Prompts
by: Raihan, Nishat, et al.
Published: (2024)

A Study on Scaling Up Multilingual News Framing Analysis
by: Akter, Syeda Sabrina, et al.
Published: (2024)

Dialect Normalization using Large Language Models and Morphological Rules
by: Dimakis, Antonios, et al.
Published: (2025)

Metadata Conditioned Large Language Models for Localization
by: Mukherjee, Anjishnu, et al.
Published: (2026)

MasonTigers at SemEval-2024 Task 1: An Ensemble Approach for Semantic Textual Relatedness
by: Goswami, Dhiman, et al.
Published: (2024)

MasonTigers at SemEval-2024 Task 9: Solving Puzzles with an Ensemble of Chain-of-Thoughts
by: Raihan, Md Nishat, et al.
Published: (2024)

Mixed-Distil-BERT: Code-mixed Language Modeling for Bangla, English, and Hindi
by: Raihan, Md Nishat, et al.
Published: (2023)

Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs
by: Mukherjee, Anjishnu, et al.
Published: (2026)

HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization
by: Peng, Qiwei, et al.
Published: (2024)

CODET: A Benchmark for Contrastive Dialectal Evaluation of Machine Translation
by: Alam, Md Mahfuz Ibn, et al.
Published: (2023)

Talent or Luck? Evaluating Attribution Bias in Large Language Models
by: Raj, Chahat, et al.
Published: (2025)

Large Language Models in Computer Science Education: A Systematic Literature Review
by: Raihan, Nishat, et al.
Published: (2024)

Speaking of Language: Reflections on Metalanguage Research in NLP
by: Schneider, Nathan, et al.
Published: (2026)

Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models
by: Mukherjee, Anjishnu, et al.
Published: (2024)

Dialectal Toxicity Detection: Evaluating LLM-as-a-Judge Consistency Across Language Varieties
by: Faisal, Fahim, et al.
Published: (2024)

Safe Language Generation in the Limit
by: Anastasopoulos, Antonios, et al.
Published: (2026)

VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models
by: Raj, Chahat, et al.
Published: (2025)

Script-Agnostic Language Identification
by: Agarwal, Milind, et al.
Published: (2024)

Back to School: Translation Using Grammar Books
by: Hus, Jonathan, et al.
Published: (2024)

GMU Systems for the IWSLT 2025 Low-Resource Speech Translation Shared Task
by: Meng, Chutong, et al.
Published: (2025)

MultiPragEval: Multilingual Pragmatic Evaluation of Large Language Models
by: Park, Dojun, et al.
Published: (2024)

Unlearning Climate Misinformation in Large Language Models
by: Fore, Michael, et al.
Published: (2024)

Data-Augmentation-Based Dialectal Adaptation for LLMs
by: Faisal, Fahim, et al.
Published: (2024)

Eka-Eval: An Evaluation Framework for Low-Resource Multilingual Large Language Models
by: Sinha, Samridhi Raj, et al.
Published: (2025)

GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models
by: Luo, Hengyu, et al.
Published: (2025)

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
by: Yu, Zhaojian, et al.
Published: (2024)

MasonTigers at SemEval-2024 Task 8: Performance Analysis of Transformer-based Models on Machine-Generated Text Detection
by: Puspo, Sadiya Sayara Chowdhury, et al.
Published: (2024)

OMGEval: An Open Multilingual Generative Evaluation Benchmark for Large Language Models
by: Liu, Yang, et al.
Published: (2024)

MasonPerplexity at Multimodal Hate Speech Event Detection 2024: Hate Speech and Target Detection Using Transformer Ensembles
by: Ganguly, Amrita, et al.
Published: (2024)

A Case Study on Filtering for End-to-End Speech Translation
by: Alam, Md Mahfuz Ibn, et al.
Published: (2024)

A Morphologically-Aware Dictionary-based Data Augmentation Technique for Machine Translation of Under-Represented Languages
by: Alam, Md Mahfuz Ibn, et al.
Published: (2024)