Imagen de Portada

CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yu, Hao, Shen, Bo, Ran, Dezhi, Zhang, Jiaxin, Zhang, Qi, Ma, Yuchi, Liang, Guangtai, Li, Ying, Wang, Qianxiang, Xie, Tao
Formato:	Preprint
Publicado:	2023
Materias:	Software Engineering
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

GraphCoder: Enhancing Repository-Level Code Completion via Code Context Graph-based Retrieval and Language Model
por: Liu, Wei, et al.
Publicado: (2024)

WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics
por: Liu, Chenxu, et al.
Publicado: (2026)

PlayCoder: Making LLM-Generated GUI Code Playable
por: Peng, Zhiyuan, et al.
Publicado: (2026)

SolEval: Benchmarking Large Language Models for Repository-level Solidity Code Generation
por: Peng, Zhiyuan, et al.
Publicado: (2025)

ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation
por: Liu, Sicong, et al.
Publicado: (2026)

On the Evaluation of Large Language Models in Unit Test Generation
por: Yang, Lin, et al.
Publicado: (2024)

SWE-bench-java: A GitHub Issue Resolving Benchmark for Java
por: Zan, Daoguang, et al.
Publicado: (2024)

ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
por: Xie, Bang, et al.
Publicado: (2026)

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
por: Dou, Shihan, et al.
Publicado: (2024)

DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models
por: Chen, Yunnong, et al.
Publicado: (2025)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
por: Chen, Yeheng, et al.
Publicado: (2026)

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
por: Zheng, Qinkai, et al.
Publicado: (2023)

CodeV: Issue Resolving with Visual Data
por: Zhang, Linhao, et al.
Publicado: (2024)

HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation
por: Zheng, Dewu, et al.
Publicado: (2024)

CodeR: Issue Resolving with Multi-Agent and Task Graphs
por: Chen, Dong, et al.
Publicado: (2024)

WybeCoder: Verified Imperative Code Generation
por: Gloeckle, Fabian, et al.
Publicado: (2026)

MaintainCoder: Maintainable Code Generation Under Dynamic Requirements
por: Wang, Zhengren, et al.
Publicado: (2025)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
por: Li, Jia, et al.
Publicado: (2024)

AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation
por: Zhu, Yueheng, et al.
Publicado: (2025)

StructCoder: Structure-Aware Transformer for Code Generation
por: Tipirneni, Sindhu, et al.
Publicado: (2022)

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
por: Zheng, Dewu, et al.
Publicado: (2024)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
por: Chou, Jason, et al.
Publicado: (2025)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
por: Paul, Debalina Ghosh, et al.
Publicado: (2024)

SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
por: Ye, Zhifan, et al.
Publicado: (2025)

HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent
por: Wu, Jie JW, et al.
Publicado: (2024)

AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
por: Jiang, Tianyue, et al.
Publicado: (2026)

FairCoder: Evaluating Social Bias of LLMs in Code Generation
por: Du, Yongkang, et al.
Publicado: (2025)

Directional Diffusion-Style Code Editing Pre-training
por: Liang, Qingyuan, et al.
Publicado: (2025)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
por: Kuhar, Sachit, et al.
Publicado: (2024)

Seed-Coder: Let the Code Model Curate Data for Itself
por: Seed, ByteDance, et al.
Publicado: (2025)

FastCoder: Accelerating Repository-level Code Generation via Efficient Retrieval and Verification
por: Zhao, Qianhui, et al.
Publicado: (2025)

SynthCoder: A Synthetical Strategy to Tune LLMs for Code Completion
por: Yu, Dongjun, et al.
Publicado: (2025)

DeCon: Detecting Incorrect Assertions via Postconditions Generated by a Large Language Model
por: Yu, Hao, et al.
Publicado: (2025)

ZeroCoder: Can LLMs Improve Code Generation Without Ground-Truth Supervision?
por: Fan, Lishui, et al.
Publicado: (2026)

On the Effect of Token Merging on Pre-trained Models for Code
por: Saad, Mootez, et al.
Publicado: (2025)

AP2O-Coder: Adaptively Progressive Preference Optimization for Reducing Compilation and Runtime Errors in LLM-Generated Code
por: Zhang, Jianqing, et al.
Publicado: (2025)

Pragmatic Reasoning improves LLM Code Generation
por: Cao, Zhuchen, et al.
Publicado: (2025)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
por: Guo, Chengquan, et al.
Publicado: (2024)

Benchmarking and Revisiting Code Generation Assessment: A Mutation-Based Approach
por: Wang, Longtian, et al.
Publicado: (2025)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
por: Wang, Jiexin, et al.
Publicado: (2024)