Image de couverture de livre

Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Paul, Debalina Ghosh, Zhu, Hong, Bayley, Ian
Format:	Preprint
Publié:	2024
Sujets:	Artificial Intelligence Software Engineering
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
par: Paul, Debalina Ghosh, et autres
Publié: (2024)

Investigating The Smells of LLM Generated Code
par: Paul, Debalina Ghosh, et autres
Publié: (2025)

Operational Robustness of LLMs on Code Generation
par: Paul, Debalina Ghosh, et autres
Publié: (2026)

Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metrics
par: Liu, Chunhua, et autres
Publié: (2025)

User Centric Evaluation of Code Generation Tools
par: Miah, Tanha, et autres
Publié: (2024)

A Qualitative Investigation into LLM-Generated Multilingual Code Comments and Automatic Evaluation Metrics
par: Katzy, Jonathan, et autres
Publié: (2025)

Code Review Agent Benchmark
par: Zhang, Yuntong, et autres
Publié: (2026)

Lyra: A Benchmark for Turducken-Style Code Generation
par: Liang, Qingyuan, et autres
Publié: (2021)

WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics
par: Liu, Chenxu, et autres
Publié: (2026)

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
par: Lian, Keke, et autres
Publié: (2025)

COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
par: Meaden, James, et autres
Publié: (2025)

CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation
par: Yang, Guang, et autres
Publié: (2025)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
par: Padwal, Vedant
Publié: (2026)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
par: Guo, Chengquan, et autres
Publié: (2024)

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
par: Zhu, Qiming, et autres
Publié: (2024)

Technique to Baseline QE Artefact Generation Aligned to Quality Metrics
par: Farchi, Eitan, et autres
Publié: (2025)

A New Benchmark for the Appropriate Evaluation of RTL Code Optimization
par: Lu, Yao, et autres
Publié: (2026)

An Evaluation of Context Length Extrapolation in Long Code via Positional Embeddings and Efficient Attention
par: Ghosh, Madhusudan, et autres
Publié: (2026)

ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
par: Hong, Jiseung, et autres
Publié: (2026)

Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach
par: Sepidband, Melika, et autres
Publié: (2025)

Can Code Evaluation Metrics Detect Code Plagiarism?
par: Ebrahim, Fahad, et autres
Publié: (2026)

Evaluating Large Language Models for Code Review
par: Cihan, Umut, et autres
Publié: (2025)

Bridging LLM-Generated Code and Requirements: Reverse Generation technique and SBC Metric for Developer Insights
par: Ponnusamy, Ahilan Ayyachamy Nadar
Publié: (2025)

Re-Evaluating Code LLM Benchmarks Under Semantic Mutation
par: Pan, Zhiyuan, et autres
Publié: (2025)

Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice
par: Hu, Ruida, et autres
Publié: (2025)

FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
par: May, Victor, et autres
Publié: (2025)

Benchmarking Correctness and Security in Multi-Turn Code Generation
par: Rawal, Ruchit, et autres
Publié: (2025)

Automated Benchmark Generation for Repository-Level Coding Tasks
par: Vergopoulos, Konstantinos, et autres
Publié: (2025)

SWE Context Bench: A Benchmark for Context Learning in Coding
par: Zhu, Jiayuan, et autres
Publié: (2026)

CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents
par: Pereira, Kristen, et autres
Publié: (2026)

Evaluating SAP Joule for Code Generation
par: Heisler, Joshua, et autres
Publié: (2025)

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation
par: Yan, Weixiang, et autres
Publié: (2023)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
par: Zhang, Tanghaoran, et autres
Publié: (2026)

Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
par: Wu, Fan, et autres
Publié: (2026)

Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges
par: Lee, Yunseo, et autres
Publié: (2025)

UA-Code-Bench: A Competitive Programming Benchmark for Evaluating LLM Code Generation in Ukrainian
par: Syromiatnikov, Mykyta, et autres
Publié: (2025)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
par: Kuhar, Sachit, et autres
Publié: (2024)

Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation
par: Cui, Yi
Publié: (2025)

Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics
par: Borg, Markus, et autres
Publié: (2026)

Understanding the Limits of Automated Evaluation for Code Review Bots in Practice
par: Karakaya, Veli, et autres
Publié: (2026)