Cover Image

User Centric Evaluation of Code Generation Tools

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Miah, Tanha, Zhu, Hong
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review
by: Paul, Debalina Ghosh, et al.
Published: (2024)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
by: Paul, Debalina Ghosh, et al.
Published: (2024)

Investigating The Smells of LLM Generated Code
by: Paul, Debalina Ghosh, et al.
Published: (2025)

From Code-Centric to Intent-Centric Software Engineering: A Reflexive Thematic Analysis of Generative AI, Agentic Systems, and Engineering Accountability
by: De La Cruz, Elyson
Published: (2026)

ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
by: Hong, Jiseung, et al.
Published: (2026)

Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metrics
by: Liu, Chunhua, et al.
Published: (2025)

CodeGenLink: A Tool to Find the Likely Origin and License of Automatically Generated Code
by: Bifolco, Daniele, et al.
Published: (2025)

Evaluating SAP Joule for Code Generation
by: Heisler, Joshua, et al.
Published: (2025)

EvolveTool-Bench: Evaluating the Quality of LLM-Generated Tool Libraries as Software Artifacts
by: Kaliyev, Alibek T., et al.
Published: (2026)

CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging
by: Li, Zhenyu, et al.
Published: (2026)

Evaluating the Energy-Efficiency of the Code Generated by LLMs
by: Islam, Md Arman, et al.
Published: (2025)

Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion?
by: Pan, Zhenyu, et al.
Published: (2024)

Holistic Evaluation of State-of-the-Art LLMs for Code Generation
by: Zhang, Le, et al.
Published: (2025)

SCoGen: Scenario-Centric Graph-Based Synthesis of Real-World Code Problems
by: Yao, Xifeng, et al.
Published: (2025)

Operational Robustness of LLMs on Code Generation
by: Paul, Debalina Ghosh, et al.
Published: (2026)

CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs
by: Basha, Manaal, et al.
Published: (2025)

Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capability
by: He, Mengliang, et al.
Published: (2025)

Class-Level Code Generation from Natural Language Using Iterative, Tool-Enhanced Reasoning over Repository
by: Deshpande, Ajinkya, et al.
Published: (2024)

Automated User Story Generation with Test Case Specification Using Large Language Model
by: Rahman, Tajmilur, et al.
Published: (2024)

Squeez: Task-Conditioned Tool-Output Pruning for Coding Agents
by: Kovács, Ádám
Published: (2026)

Applying an Agentic Coding Tool for Improving Published Algorithm Implementations
by: Suwannik, Worasait
Published: (2026)

ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents
by: Li, Dawei, et al.
Published: (2026)

Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks
by: Hyun, Sangwon, et al.
Published: (2025)

WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models
by: Lei, Xinping, et al.
Published: (2026)

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
by: Hu, Ruida, et al.
Published: (2026)

CodeGolf Bench: A Multi-Language Benchmark for Evaluating Concise Code Generation Capabilities of Large Language Models
by: Padwal, Vedant
Published: (2026)

SEMAG: Self-Evolutionary Multi-Agent Code Generation
by: Peng, Yulin, et al.
Published: (2026)

Lyra: A Benchmark for Turducken-Style Code Generation
by: Liang, Qingyuan, et al.
Published: (2021)

LLMs in Web Development: Evaluating LLM-Generated PHP Code Unveiling Vulnerabilities and Limitations
by: Tóth, Rebeka, et al.
Published: (2024)

InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation
by: Chen, Qiaosheng, et al.
Published: (2025)

COMPASS: A Multi-Dimensional Benchmark for Evaluating Code Generation in Large Language Models
by: Meaden, James, et al.
Published: (2025)

CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generation
by: Xiong, Zhanhang, et al.
Published: (2025)

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
by: Lian, Keke, et al.
Published: (2025)

Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
by: Christakis, Nicholas, et al.
Published: (2025)

A Qualitative Investigation into LLM-Generated Multilingual Code Comments and Automatic Evaluation Metrics
by: Katzy, Jonathan, et al.
Published: (2025)

Reverse Engineering User Stories from Code using Large Language Models
by: Ouf, Mohamed, et al.
Published: (2025)

AI-Generated Smells: An Analysis of Code and Architecture in LLM and Agent-Driven Development
by: Zhu, Yuecai, et al.
Published: (2026)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
by: Guo, Chengquan, et al.
Published: (2024)

Towards Advancing Code Generation with Large Language Models: A Research Roadmap
by: Jin, Haolin, et al.
Published: (2025)

DynamicsLLM: a Dynamic Analysis-based Tool for Generating Intelligent Execution Traces Using LLMs to Detect Android Behavioural Code Smells
by: Cherief, Houcine Abdelkader, et al.
Published: (2026)