Cover Image

HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Peng, Qiwei, Chai, Yekun, Li, Xuhong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Programming Languages Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
by: Yu, Zhaojian, et al.
Published: (2024)

NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts
by: Zhang, Shudan, et al.
Published: (2024)

IndicEval-XL: Bridging Linguistic Diversity in Code Generation Across Indic Languages
by: Singh, Ujjwal, et al.
Published: (2025)

SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation
by: Petrukha, Ivan, et al.
Published: (2025)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
by: Chen, Yeheng, et al.
Published: (2026)

Benchmarking LLM Code Generation for Audio Programming with Visual Dataflow Languages
by: Zhang, William, et al.
Published: (2024)

Generating Equivalent Representations of Code By A Self-Reflection Approach
by: Li, Jia, et al.
Published: (2024)

Finding Compiler Bugs through Cross-Language Code Generator and Differential Testing
by: Feng, Qiong, et al.
Published: (2025)

Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM
by: Xia, Chunqiu Steven, et al.
Published: (2024)

A Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks
by: Dandamudi, Rohit, et al.
Published: (2024)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluation
by: Liu, Jiaheng, et al.
Published: (2024)

PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Models
by: Chen, Simin, et al.
Published: (2024)

Mix-of-Language-Experts Architecture for Multilingual Programming
by: Zong, Yifan, et al.
Published: (2025)

PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization
by: Jing, Huihao, et al.
Published: (2026)

ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
by: Liu, Kaiyuan, et al.
Published: (2025)

Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
by: Li, Zhuohao, et al.
Published: (2025)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
by: Wang, Jiexin, et al.
Published: (2024)

SWE-QA: Can Language Models Answer Repository-level Code Questions?
by: Peng, Weihan, et al.
Published: (2025)

PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback
by: Peng, Yun, et al.
Published: (2024)

Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval
by: Wu, Jiarong, et al.
Published: (2025)

Debiasing Multilingual LLMs in Cross-lingual Latent Space
by: Peng, Qiwei, et al.
Published: (2025)

How Programming Concepts and Neurons Are Shared in Code Language Models
by: Kargaran, Amir Hossein, et al.
Published: (2025)

Strengthening Programming Comprehension in Large Language Models through Code Generation
by: Ren, Xiaoning, et al.
Published: (2025)

Semantic Source Code Segmentation using Small and Large Language Models
by: Dahou, Abdelhalim, et al.
Published: (2025)

SmartEval: A Benchmark for Evaluating LLM-Generated Smart Contracts from Natural Language Specifications
by: Goel, Abhinav, et al.
Published: (2026)

Structural Code Search using Natural Language Queries
by: Limpanukorn, Ben, et al.
Published: (2025)

Dafny as Verification-Aware Intermediate Language for Code Generation
by: Li, Yue Chen, et al.
Published: (2025)

From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
by: Shi, Yuling, et al.
Published: (2024)

Multilingual Multimodal Software Developer for Code Generation
by: Chai, Linzheng, et al.
Published: (2025)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
by: Chou, Jason, et al.
Published: (2025)

Is Self-Repair a Silver Bullet for Code Generation?
by: Olausson, Theo X., et al.
Published: (2023)

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?
by: Zhao, Yuwei, et al.
Published: (2024)

iScript: A Domain-Adapted Large Language Model and Benchmark for Physical Design Tcl Script Generation
by: Xu, Ning, et al.
Published: (2026)

Pareto Optimal Code Generation
by: Orlanski, Gabriel, et al.
Published: (2025)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

COBOL-Coder: Domain-Adapted Large Language Models for COBOL Code Generation and Translation
by: Dau, Anh T. V., et al.
Published: (2026)

Python Symbolic Execution with LLM-powered Code Generation
by: Wang, Wenhan, et al.
Published: (2024)

AdaCCD: Adaptive Semantic Contrasts Discovery Based Cross Lingual Adaptation for Code Clone Detection
by: Du, Yangkai, et al.
Published: (2023)