Cover Image

From Output to Evaluation: Does Raw Instruction-Tuned Code LLMs Output Suffice for Fill-in-the-Middle Code Generation?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ahmad, Wasi Uddin, Majumdar, Somshubra, Ginsburg, Boris
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs
by: Ahmad, Wasi Uddin, et al.
Published: (2025)

From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents
by: Ludwig, Nikolai, et al.
Published: (2026)

Scoring Verifiers: Evaluating Synthetic Verification for Code and Reasoning
by: Ficek, Aleksander, et al.
Published: (2025)

Repoformer: Selective Retrieval for Repository-Level Code Completion
by: Wu, Di, et al.
Published: (2024)

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
by: Gong, Linyuan, et al.
Published: (2024)

DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode
by: Han, Hojae, et al.
Published: (2026)

Evaluation of Code LLMs on Geospatial Code Generation
by: Gramacki, Piotr, et al.
Published: (2024)

Structure-Aware Fill-in-the-Middle Pretraining for Code
by: Gong, Linyuan, et al.
Published: (2025)

To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing
by: Cheng, Wei, et al.
Published: (2026)

OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
by: Ahmad, Wasi Uddin, et al.
Published: (2025)

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct
by: Wu, Yutong, et al.
Published: (2024)

FairCoder: Evaluating Social Bias of LLMs in Code Generation
by: Du, Yongkang, et al.
Published: (2025)

From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning
by: Zhang, Jiajun, et al.
Published: (2026)

InstructCoder: Instruction Tuning Large Language Models for Code Editing
by: Li, Kaixin, et al.
Published: (2023)

Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models
by: Majumdar, Somshubra, et al.
Published: (2024)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
by: Yang, Jialin, et al.
Published: (2025)

Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey
by: Wang, Junqiao, et al.
Published: (2024)

Grounding Data Science Code Generation with Input-Output Specifications
by: Wen, Yeming, et al.
Published: (2024)

Code-Vision: Evaluating Multimodal LLMs Logic Understanding and Code Generation Capabilities
by: Wang, Hanbin, et al.
Published: (2025)

How Do Your Code LLMs Perform? Empowering Code Instruction Tuning with High-Quality Data
by: Wang, Yejie, et al.
Published: (2024)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
by: Wang, Jiexin, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation
by: Ni, Yuansheng, et al.
Published: (2025)

AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions
by: Wang, Zihan, et al.
Published: (2025)

Showing LLM-Generated Code Selectively Based on Confidence of LLMs
by: Li, Jia, et al.
Published: (2024)

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
by: Zhuo, Terry Yue, et al.
Published: (2024)

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation
by: Yan, Weixiang, et al.
Published: (2023)

Evaluating and Achieving Controllable Code Completion in Code LLM
by: Zhang, Jiajun, et al.
Published: (2026)

CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
by: Tian, Yuchen, et al.
Published: (2024)

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation
by: Chen, Zaoyu, et al.
Published: (2026)

Bridging Developer Instructions and Code Completion Through Instruction-Aware Fill-in-the-Middle Paradigm
by: Sun, Zhensu, et al.
Published: (2025)

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
by: Zhuo, Terry Yue, et al.
Published: (2024)

Code Fingerprints: Disentangled Attribution of LLM-Generated Code
by: Guo, Jiaxun, et al.
Published: (2026)

VersiCode: Towards Version-controllable Code Generation
by: Wu, Tongtong, et al.
Published: (2024)

From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation
by: Bui, Minh Duc, et al.
Published: (2026)

Humanity's Last Code Exam: Can Advanced LLMs Conquer Human's Hardest Code Competition?
by: Li, Xiangyang, et al.
Published: (2025)

Eliciting Instruction-tuned Code Language Models' Capabilities to Utilize Auxiliary Function for Code Generation
by: Lee, Seonghyeon, et al.
Published: (2024)

CodeRAG-Bench: Can Retrieval Augment Code Generation?
by: Wang, Zora Zhiruo, et al.
Published: (2024)

Terminal Agents Suffice for Enterprise Automation
by: Bechard, Patrice, et al.
Published: (2026)

WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning
by: Yu, Zhaojian, et al.
Published: (2023)