Cover Image

Edit, But Verify: An Empirical Audit of Instructed Code-Editing Benchmarks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ebrahimi, Amir M., Rajbahadur, Gopi Krishnan
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality
by: Li, Hao, et al.
Published: (2024)

Permissive-Washing in the Open AI Supply Chain: A Large-Scale Audit of License Integrity
by: Jewitt, James, et al.
Published: (2026)

From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI Ecosystem
by: Jewitt, James, et al.
Published: (2025)

Data Quality Antipatterns for Software Analytics
by: Bhatia, Aaditya, et al.
Published: (2024)

From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap
by: Rajbahadur, Gopi Krishnan, et al.
Published: (2024)

Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate Overfitting
by: Li, Hao, et al.
Published: (2024)

SimClone: Detecting Tabular Data Clones using Value Similarity
by: Yang, Xu, et al.
Published: (2024)

LicenseGPT: A Fine-tuned Foundation Model for Publicly Available Dataset License Compliance
by: Tan, Jingwen, et al.
Published: (2024)

SAFEdit: Does Multi-Agent Decomposition Resolve the Reliability Challenges of Instructed Code Editing?
by: Tarshish, Noam, et al.
Published: (2026)

SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation
by: Oliva, Gustavo A., et al.
Published: (2025)

An Execution-Verified Multi-Language Benchmark for Code Semantic Reasoning
by: Li, Yikun, et al.
Published: (2026)

Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware
by: Hassan, Ahmed E., et al.
Published: (2024)

AutoCoder: Enhancing Code Large Language Model with \textsc{AIEV-Instruct}
by: Lei, Bin, et al.
Published: (2024)

Learning Performance-Improving Code Edits
by: Shypula, Alexander, et al.
Published: (2023)

Magicoder: Empowering Code Generation with OSS-Instruct
by: Wei, Yuxiang, et al.
Published: (2023)

WybeCoder: Verified Imperative Code Generation
by: Gloeckle, Fabian, et al.
Published: (2026)

SIEVE: Towards Verifiable Certification for Code-datasets
by: Mbodji, Fatou Ndiaye, et al.
Published: (2025)

Implementing AI Bill of Materials (AI BOM) with SPDX 3.0: A Comprehensive Guide to Creating AI and Dataset Bill of Materials
by: Bennet, Karen, et al.
Published: (2025)

Needle in the Repo: A Benchmark for Maintainability in AI-Generated Repository Edits
by: Zhu, Haichao, et al.
Published: (2026)

Aletheia: What Makes RLVR For Code Verifiers Tick?
by: Venkatkrishna, Vatsal, et al.
Published: (2026)

VeriContest: A Competitive-Programming Benchmark for Verifiable Code Generation
by: Xie, Zichen, et al.
Published: (2026)

AlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithms
by: Zhao, Haoyu, et al.
Published: (2026)

Code4MeV2: a Research-oriented Code-completion Platform
by: Koohestani, Roham, et al.
Published: (2025)

ICE-Score: Instructing Large Language Models to Evaluate Code
by: Zhuo, Terry Yue
Published: (2023)

Verifying LLM-Generated Code in the Context of Software Verification with Ada/SPARK
by: Cramer, Marcos, et al.
Published: (2025)

Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
by: Cassano, Federico, et al.
Published: (2023)

An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications
by: Hasan, Mohammed Mehedi, et al.
Published: (2025)

Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation
by: Vartziotis, Tina, et al.
Published: (2024)

CVeDRL: An Efficient Code Verifier via Difficulty-aware Reinforcement Learning
by: Shi, Ji, et al.
Published: (2026)

Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications
by: Jin, Haolin, et al.
Published: (2025)

Let the Code LLM Edit Itself When You Edit the Code
by: He, Zhenyu, et al.
Published: (2024)

From Benchmark Data To Applicable Program Repair: An Experience Report
by: Chandramohan, Mahinthan, et al.
Published: (2025)

An Empirical Study of Knowledge Distillation for Code Understanding Tasks
by: Wang, Ruiqi, et al.
Published: (2025)

From Natural Language to Verified Code: Toward AI Assisted Problem-to-Code Generation with Dafny-Based Formal Verification
by: Erfan, Md, et al.
Published: (2026)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
by: Guo, Chengquan, et al.
Published: (2024)

AutoICE: Automatically Synthesizing Verifiable C Code via LLM-driven Evolution
by: Luo, Weilin, et al.
Published: (2025)

Code Review Agent Benchmark
by: Zhang, Yuntong, et al.
Published: (2026)

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct
by: Wu, Yutong, et al.
Published: (2024)

The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
by: Vasilevski, Kirill, et al.
Published: (2025)

Bin2Vec: Interpretable and Auditable Multi-View Binary Analysis for Code Plagiarism Detection
by: Moussaoui, Moussa, et al.
Published: (2025)