Copertina

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Paul, Debalina Ghosh, Zhu, Hong, Bayley, Ian
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Software Engineering Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review
di: Paul, Debalina Ghosh, et al.
Pubblicazione: (2024)

Investigating The Smells of LLM Generated Code
di: Paul, Debalina Ghosh, et al.
Pubblicazione: (2025)

Operational Robustness of LLMs on Code Generation
di: Paul, Debalina Ghosh, et al.
Pubblicazione: (2026)

LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation
di: Kuhar, Sachit, et al.
Pubblicazione: (2024)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
di: Zhang, Tanghaoran, et al.
Pubblicazione: (2026)

User Centric Evaluation of Code Generation Tools
di: Miah, Tanha, et al.
Pubblicazione: (2024)

DevEval: Evaluating Code Generation in Practical Software Projects
di: Li, Jia, et al.
Pubblicazione: (2024)

CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection
di: Zhang, Qingyu, et al.
Pubblicazione: (2025)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
di: Wu, Qinyun, et al.
Pubblicazione: (2024)

MLDebugging: Towards Benchmarking Code Debugging Across Multi-Library Scenarios
di: Huang, Jinyang, et al.
Pubblicazione: (2025)

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
di: Zheng, Qinkai, et al.
Pubblicazione: (2023)

LiCoEval: Evaluating LLMs on License Compliance in Code Generation
di: Xu, Weiwei, et al.
Pubblicazione: (2024)

ContractEval: A Benchmark for Evaluating Contract-Satisfying Assertions in Code Generation
di: Lim, Soohan, et al.
Pubblicazione: (2025)

Lyra: A Benchmark for Turducken-Style Code Generation
di: Liang, Qingyuan, et al.
Pubblicazione: (2021)

TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models
di: Tambon, Florian, et al.
Pubblicazione: (2024)

VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation
di: Vijayaraghavan, Prashanth, et al.
Pubblicazione: (2024)

Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation
di: Hamedi, Parisa, et al.
Pubblicazione: (2025)

DecompileBench: A Comprehensive Benchmark for Evaluating Decompilers in Real-World Scenarios
di: Gao, Zeyu, et al.
Pubblicazione: (2025)

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
di: Lian, Keke, et al.
Pubblicazione: (2025)

COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
di: Meaden, James, et al.
Pubblicazione: (2025)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
di: Padwal, Vedant
Pubblicazione: (2026)

DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents
di: Hong, Sirui, et al.
Pubblicazione: (2026)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
di: Guo, Chengquan, et al.
Pubblicazione: (2024)

DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
di: Zhu, Qiming, et al.
Pubblicazione: (2024)

A New Benchmark for the Appropriate Evaluation of RTL Code Optimization
di: Lu, Yao, et al.
Pubblicazione: (2026)

An Evaluation of Context Length Extrapolation in Long Code via Positional Embeddings and Efficient Attention
di: Ghosh, Madhusudan, et al.
Pubblicazione: (2026)

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
di: Hu, Ruida, et al.
Pubblicazione: (2026)

ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
di: Hong, Jiseung, et al.
Pubblicazione: (2026)

CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding
di: Luo, Hanjun, et al.
Pubblicazione: (2025)

Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metrics
di: Liu, Chunhua, et al.
Pubblicazione: (2025)

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents
di: Lu, Qinghua, et al.
Pubblicazione: (2025)

Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows
di: Li, Chenxin, et al.
Pubblicazione: (2026)

TaskEval: Synthesised Evaluation for Foundation-Model Tasks
di: Widanapathiranage, Dilani, et al.
Pubblicazione: (2025)

SCoGen: Scenario-Centric Graph-Based Synthesis of Real-World Code Problems
di: Yao, Xifeng, et al.
Pubblicazione: (2025)

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages
di: Kammakomati, Mehant, et al.
Pubblicazione: (2024)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
di: Yang, Jialin, et al.
Pubblicazione: (2025)

Re-Evaluating Code LLM Benchmarks Under Semantic Mutation
di: Pan, Zhiyuan, et al.
Pubblicazione: (2025)

FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
di: May, Victor, et al.
Pubblicazione: (2025)

Benchmarking Correctness and Security in Multi-Turn Code Generation
di: Rawal, Ruchit, et al.
Pubblicazione: (2025)

Automated Benchmark Generation for Repository-Level Coding Tasks
di: Vergopoulos, Konstantinos, et al.
Pubblicazione: (2025)