Cover Image

ATime-Consistent Benchmark for Repository-Level Software Engineering Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xianpeng, Sun, Sun, Haonan, Yu, Tian, Ma, Sheng, Zhang, Qincheng, Rao, Lifei, Tian, Chen
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
by: Lian, Keke, et al.
Published: (2025)

Structurally Aligned Subtask-Level Memory for Software Engineering Agents
by: Shen, Kangning, et al.
Published: (2026)

Towards Comprehensive Benchmarking Infrastructure for LLMs In Software Engineering
by: Rodriguez-Cardenas, Daniel, et al.
Published: (2026)

RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph
by: Ouyang, Siru, et al.
Published: (2024)

Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
by: Kuang, Jiayi, et al.
Published: (2025)

Automated Benchmark Generation for Repository-Level Coding Tasks
by: Vergopoulos, Konstantinos, et al.
Published: (2025)

LLM-Assisted Repository-Level Generation with Structured Spec-Driven Engineering
by: Feng, Shuzhao, et al.
Published: (2026)

RAG-Verus: Repository-Level Program Verification with LLMs using Retrieval Augmented Generation
by: Zhong, Sicheng, et al.
Published: (2025)

Toward Scalable Automated Repository-Level Datasets for Software Vulnerability Detection
by: Lbath, Amine
Published: (2026)

AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Context
by: Zhang, Lei, et al.
Published: (2026)

Beyond Code Snippets: Benchmarking LLMs on Repository-Level Question Answering
by: Alebachew, Yoseph Berhanu, et al.
Published: (2026)

Mining Software Repositories for Expert Recommendation
by: Marshall, Chad, et al.
Published: (2025)

ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
by: Hong, Jiseung, et al.
Published: (2026)

From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level
by: Li, Jia, et al.
Published: (2026)

Skeleton-Guided-Translation: A Benchmarking Framework for Code Repository Translation with Fine-Grained Quality Evaluation
by: Zhang, Xing, et al.
Published: (2025)

Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
by: Wang, Xinchen, et al.
Published: (2026)

Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
by: Gloaguen, Thibaud, et al.
Published: (2026)

REPOFUSE: Repository-Level Code Completion with Fused Dual Context
by: Liang, Ming, et al.
Published: (2024)

GREPO: A Benchmark for Graph Neural Networks on Repository-Level Bug Localization
by: Wang, Juntong, et al.
Published: (2026)

AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
by: Jiang, Tianyue, et al.
Published: (2026)

Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents
by: Zhao, Chenyu, et al.
Published: (2026)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

OmniCode: A Benchmark for Evaluating Software Engineering Agents
by: Sonwane, Atharv, et al.
Published: (2026)

Persistent Cross-Attempt State Optimization for Repository-Level Code Generation
by: Pan, Ruwei, et al.
Published: (2026)

On the Impacts of Contexts on Repository-Level Code Generation
by: Hai, Nam Le, et al.
Published: (2024)

Toward Executable Repository-Level Code Generation via Environment Alignment
by: Pan, Ruwei, et al.
Published: (2026)

A Benchmark for Localizing Code and Non-Code Issues in Software Projects
by: Zhang, Zejun, et al.
Published: (2025)

CodeClash: Benchmarking Goal-Oriented Software Engineering
by: Yang, John, et al.
Published: (2025)

Unified Software Engineering Agent as AI Software Engineer
by: Applis, Leonhard, et al.
Published: (2025)

SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
by: Wang, Lilin, et al.
Published: (2025)

In-Context Code-Text Learning for Bimodal Software Engineering
by: Tang, Xunzhu, et al.
Published: (2024)

Needle in the Repo: A Benchmark for Maintainability in AI-Generated Repository Edits
by: Zhu, Haichao, et al.
Published: (2026)

SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?
by: Han, Tingxu, et al.
Published: (2026)

Software Performance Engineering for Foundation Model-Powered Software
by: Zhang, Haoxiang, et al.
Published: (2024)

RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models
by: Liu, Jingjing, et al.
Published: (2025)

GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
by: Ni, Ziyi, et al.
Published: (2025)

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
by: Chen, Zhi, et al.
Published: (2026)

SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding
by: Cai, Songcheng, et al.
Published: (2026)

RepoReviewer: A Local-First Multi-Agent Architecture for Repository-Level Code Review
by: Zhang, Peng
Published: (2026)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
by: Duston, Titouan, et al.
Published: (2025)