Cover Image

Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xia, Chunqiu Steven, Deng, Yinlin, Zhang, Lingming
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Computation and Language Machine Learning Programming Languages
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Agentless: Demystifying LLM-based Software Engineering Agents
by: Xia, Chunqiu Steven, et al.
Published: (2024)

SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation
by: Petrukha, Ivan, et al.
Published: (2025)

Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
by: Xia, Chunqiu Steven, et al.
Published: (2025)

WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models
by: Yang, Chenyuan, et al.
Published: (2023)

NExT: Teaching Large Language Models to Reason about Code Execution
by: Ni, Ansong, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
by: Li, Zhuohao, et al.
Published: (2025)

HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization
by: Peng, Qiwei, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

How Natural Language Proficiency Shapes GenAI Code for Software Engineering Tasks
by: Rojpaisarnkit, Ruksit, et al.
Published: (2025)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
by: Zheng, Dewu, et al.
Published: (2024)

Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT
by: Xia, Chunqiu Steven, et al.
Published: (2023)

StackEval: Benchmarking LLMs in Coding Assistance
by: Shah, Nidhish, et al.
Published: (2024)

TokDrift: When LLM Speaks in Subwords but Code Speaks in Grammar
by: Li, Yinxi, et al.
Published: (2025)

PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization
by: Jing, Huihao, et al.
Published: (2026)

PurpCode: Reasoning for Safer Code Generation
by: Liu, Jiawei, et al.
Published: (2025)

Correctness-Guaranteed Code Generation via Constrained Decoding
by: Li, Lingxiao, et al.
Published: (2025)

Revisiting the Plastic Surgery Hypothesis via Large Language Models
by: Xia, Chunqiu Steven, et al.
Published: (2023)

Benchmarking LLM Code Generation for Audio Programming with Visual Dataflow Languages
by: Zhang, William, et al.
Published: (2024)

SynCode: LLM Generation with Grammar Augmentation
by: Ugare, Shubham, et al.
Published: (2024)

Evaluating Language Models for Efficient Code Generation
by: Liu, Jiawei, et al.
Published: (2024)

Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning
by: Xia, Chunqiu Steven, et al.
Published: (2022)

Neural Models for Source Code Synthesis and Completion
by: Niyogi, Mitodru
Published: (2024)

Python Symbolic Execution with LLM-powered Code Generation
by: Wang, Wenhan, et al.
Published: (2024)

A Multi-Perspective Architecture for Semantic Code Search
by: Haldar, Rajarshi, et al.
Published: (2020)

SelfCodeAlign: Self-Alignment for Code Generation
by: Wei, Yuxiang, et al.
Published: (2024)

Automatically Testing Functional Properties of Code Translation Models
by: Eniser, Hasan Ferit, et al.
Published: (2023)

Language Models for Code Completion: A Practical Evaluation
by: Izadi, Maliheh, et al.
Published: (2024)

Defusing Logic Bombs in Symbolic Execution with LLM-Generated Ghost Code
by: Bouras, Dimitrios Stamatios, et al.
Published: (2026)

NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts
by: Zhang, Shudan, et al.
Published: (2024)

Neural Code Translation of Legacy Code: APL to C#
by: Ramadan, Abdulrahman, et al.
Published: (2026)

Benchmarking LLM for Code Smells Detection: OpenAI GPT-4.0 vs DeepSeek-V3
by: Sadik, Ahmed R., et al.
Published: (2025)

Finding Missed Code Size Optimizations in Compilers using LLMs
by: Italiano, Davide, et al.
Published: (2024)

Large Language Models for Multilingual Code Intelligence: A Survey
by: Jiang, Chao, et al.
Published: (2026)

Grounding Data Science Code Generation with Input-Output Specifications
by: Wen, Yeming, et al.
Published: (2024)

EffiPair: Improving the Efficiency of LLM-generated Code with Relative Contrastive Feedback
by: Hajizadeh, Samira, et al.
Published: (2026)

COBOLAssist: Analyzing and Fixing Compilation Errors for LLM-Powered COBOL Code Generation
by: Dau, Anh T. V., et al.
Published: (2026)

Constrained Decoding for Fill-in-the-Middle Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars
by: Melcer, Daniel, et al.
Published: (2024)

SIMCOPILOT: Evaluating Large Language Models for Copilot-Style Code Generation
by: Jiang, Mingchao, et al.
Published: (2025)

Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing
by: Wei, Jiayi, et al.
Published: (2023)