Cover Image

ExecRepoBench: Multi-level Executable Code Completion Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Jian, Zhang, Jiajun, Yang, Jiaxi, Jin, Ke, Zhang, Lei, Peng, Qiyao, Deng, Ken, Miao, Yibo, Liu, Tianyu, Cui, Zeyu, Hui, Binyuan, Lin, Junyang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Evaluating and Achieving Controllable Code Completion in Code LLM
by: Zhang, Jiajun, et al.
Published: (2026)

Evaluating and Aligning CodeLLMs on Human Preference
by: Yang, Jian, et al.
Published: (2024)

Multi-Agent Collaboration for Multilingual Code Instruction Tuning
by: Yang, Jian, et al.
Published: (2025)

SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner
by: Zhang, Lei, et al.
Published: (2025)

Turning the Tide: Repository-based Code Reflection
by: Zhang, Wei, et al.
Published: (2025)

From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning
by: Zhang, Jiajun, et al.
Published: (2026)

PlotCraft: Pushing the Limits of LLMs for Complex and Interactive Data Visualization
by: Zhang, Jiajun, et al.
Published: (2025)

CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings
by: Quan, Shanghaoran, et al.
Published: (2025)

Synthesizing Text-to-SQL Data from Weak and Strong LLMs
by: Yang, Jiaxi, et al.
Published: (2024)

SWE-RM: Execution-free Feedback For Software Engineering Agents
by: Shum, KaShun, et al.
Published: (2025)

Scaling Agentic Verifier for Competitive Coding
by: Ma, Zeyao, et al.
Published: (2026)

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Towards Better Correctness and Efficiency in Code Generation
by: Feng, Yunlong, et al.
Published: (2025)

SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
by: Shen, Chihao, et al.
Published: (2025)

Qwen2.5-Coder Technical Report
by: Hui, Binyuan, et al.
Published: (2024)

EVM-QuestBench: An Execution-Grounded Benchmark for Natural-Language Transaction Code Generation
by: Yang, Pei, et al.
Published: (2026)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)

RepoShapley: Shapley-Enhanced Context Filtering for Repository-Level Code Completion
by: Huo, Yu, et al.
Published: (2026)

ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning
by: Tang, Lingxiao, et al.
Published: (2026)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
by: Wu, Qinyun, et al.
Published: (2024)

RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices
by: Li, Jia, et al.
Published: (2026)

NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
by: Ding, Jingzhe, et al.
Published: (2025)

IFEvalCode: Controlled Code Generation
by: Yang, Jian, et al.
Published: (2025)

Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks
by: Pasquini, Dario, et al.
Published: (2024)

SummExecEdit: A Factual Consistency Benchmark in Summarization with Executable Edits
by: Thorat, Onkar, et al.
Published: (2024)

START: Self-taught Reasoner with Tools
by: Li, Chengpeng, et al.
Published: (2025)

Neurosymbolic Repo-level Code Localization
by: Xu, Xiufeng, et al.
Published: (2026)

Repo2Run: Automated Building Executable Environment for Code Repository at Scale
by: Hu, Ruida, et al.
Published: (2025)

SecCodeBench-V2 Technical Report
by: Chen, Longfei, et al.
Published: (2026)

MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Era
by: Zhang, Lei, et al.
Published: (2026)

RepoQA: Evaluating Long Context Code Understanding
by: Liu, Jiawei, et al.
Published: (2024)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

Pull Requests as a Training Signal for Repo-Level Code Editing
by: Zhu, Qinglin, et al.
Published: (2026)

CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Models
by: Zhang, Alexander, et al.
Published: (2025)

IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Web
by: Guo, Hongcheng, et al.
Published: (2024)

RepoHyper: Search-Expand-Refine on Semantic Graphs for Repository-Level Code Completion
by: Phan, Huy N., et al.
Published: (2024)

Qwen3-Coder-Next Technical Report
by: Cao, Ruisheng, et al.
Published: (2026)

Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs
by: Zhang, Lei, et al.
Published: (2024)

RepoMod-Bench: A Benchmark for Code Repository Modernization via Implementation-Agnostic Testing
by: Li, Xuefeng, et al.
Published: (2026)

SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits
by: Lin, Edward, et al.
Published: (2026)