Cover Image

The First Prompt Counts the Most! An Evaluation of Large Language Models on Iterative Example-Based Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fu, Yingjie, Li, Bozhou, Li, Linyi, Zhang, Wentao, Xie, Tao
Format:	Preprint
Published:	2024
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models
by: Li, Linyi, et al.
Published: (2024)

SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation
by: He, Yibo, et al.
Published: (2025)

Evaluating Large Language Models for Code Translation: Effects of Prompt Language and Prompt Design
by: Aljagthami, Aamer, et al.
Published: (2025)

CodeMEM: AST-Guided Adaptive Memory for Repository-Level Iterative Code Generation
by: Wang, Peiding, et al.
Published: (2026)

Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization
by: Midolo, Alessandro, et al.
Published: (2026)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
by: Wang, Peiding, et al.
Published: (2025)

Ensembling Large Language Models for Code Vulnerability Detection: An Empirical Evaluation
by: Sun, Zhihong, et al.
Published: (2025)

WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics
by: Liu, Chenxu, et al.
Published: (2026)

CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generation
by: Wang, Sizhe, et al.
Published: (2025)

A Survey on Evaluating Large Language Models in Code Generation Tasks
by: Chen, Liguo, et al.
Published: (2024)

What Builds Effective In-Context Examples for Code Generation?
by: Li, Dongze, et al.
Published: (2025)

ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generation
by: Jiang, Xue, et al.
Published: (2024)

GeoJSEval: An Automated Evaluation Framework for Large Language Models on JavaScript-Based Geospatial Computation and Visualization Code Generation
by: Chen, Guanyu, et al.
Published: (2025)

Can Large Language Models Generate Geospatial Code?
by: Hou, Shuyang, et al.
Published: (2024)

An Exploratory Study of Bayesian Prompt Optimization for Test-Driven Code Generation with Large Language Models
by: Tomar, Shlok, et al.
Published: (2025)

CFCEval: Evaluating Security Aspects in Code Generated by Large Language Models
by: Cheng, Cheng, et al.
Published: (2025)

Self-planning Code Generation with Large Language Models
by: Jiang, Xue, et al.
Published: (2023)

ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation
by: Xue, Pengyu, et al.
Published: (2024)

A Preliminary Study on the Robustness of Code Generation by Large Language Models
by: Li, Zike, et al.
Published: (2025)

From Evaluation to Enhancement: Large Language Models for Zero-Knowledge Proof Code Generation
by: Xue, Zhantong, et al.
Published: (2025)

Large Language Model Unlearning for Source Code
by: Jiang, Xue, et al.
Published: (2025)

Knowledge-Aware Code Generation with Large Language Models
by: Huang, Tao, et al.
Published: (2024)

Are Prompts All You Need? Evaluating Prompt-Based Large Language Models (LLM)s for Software Requirements Classification
by: Binkhonain, Manal, et al.
Published: (2025)

CrossPL: Evaluating Large Language Models on Cross Programming Language Code Generation
by: Xiong, Zhanhang, et al.
Published: (2025)

An Empirical Study on Influence-Based Pretraining Data Selection for Code Large Language Models
by: Xing, Chengli, et al.
Published: (2026)

Toward Green Code: Prompting Small Language Models for Energy-Efficient Code Generation
by: Ashraf, Humza, et al.
Published: (2025)

Evaluating the Impact of Post-Training Quantization on Large Language Models for Code Generation
by: Giagnorio, Alessandro, et al.
Published: (2025)

Can Large Language Models Serve as Evaluators for Code Summarization?
by: Wu, Yang, et al.
Published: (2024)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
by: Wang, Jiexin, et al.
Published: (2024)

CodeJudge: Evaluating Code Generation with Large Language Models
by: Tong, Weixi, et al.
Published: (2024)

From Restructuring to Stabilization: A Large-Scale Experiment on Iterative Code Readability Refactoring with Large Language Models
by: Peitek, Norman, et al.
Published: (2026)

Beyond Autoregression: An Empirical Study of Diffusion Large Language Models for Code Generation
by: Li, Chengze, et al.
Published: (2025)

V-GameGym: Visual Game Generation for Code Large Language Models
by: Zhang, Wei, et al.
Published: (2025)

Prompt Alchemy: Automatic Prompt Refinement for Enhancing Code Generation
by: Ye, Sixiang, et al.
Published: (2025)

Cross-Task Benchmarking and Evaluation of General-Purpose and Code-Specific Large Language Models
by: Das, Gunjan, et al.
Published: (2025)

Counting the Trees in the Forest: Evaluating Prompt Segmentation for Classifying Code Comprehension Level
by: Smith IV, David H., et al.
Published: (2025)

SpecEval: Evaluating Code Comprehension in Large Language Models via Program Specifications
by: Ma, Lezhi, et al.
Published: (2024)

Fine-Tuning and Prompt Engineering for Large Language Models-based Code Review Automation
by: Pornprasit, Chanathip, et al.
Published: (2024)

LONGCODEU: Benchmarking Long-Context Language Models on Long Code Understanding
by: Li, Jia, et al.
Published: (2025)

Fixing Large Language Models' Specification Misunderstanding for Better Code Generation
by: Tian, Zhao, et al.
Published: (2023)