Cover Image

From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Jia, Su, Yuxin, Lyu, Michael R.
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
by: Ni, Ziyi, et al.
Published: (2025)

ComBench: A Repo-level Real-world Benchmark for Compilation Error Repair
by: Li, Jia, et al.
Published: (2026)

Automated Benchmark Generation for Repository-Level Coding Tasks
by: Vergopoulos, Konstantinos, et al.
Published: (2025)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
by: Yang, Jie, et al.
Published: (2026)

Beyond Code Snippets: Benchmarking LLMs on Repository-Level Question Answering
by: Alebachew, Yoseph Berhanu, et al.
Published: (2026)

Agentic Harness for Real-World Compilers
by: Zheng, Yingwei, et al.
Published: (2026)

CodeSense: a Real-World Benchmark and Dataset for Code Semantic Reasoning
by: Roy, Monoshi Kumar, et al.
Published: (2025)

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
by: Lian, Keke, et al.
Published: (2025)

RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations
by: Li, Hanyu, et al.
Published: (2026)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
by: Wu, Qinyun, et al.
Published: (2024)

Hierarchical Repository-Level Code Summarization for Business Applications Using Local LLMs
by: Dhulshette, Nilesh, et al.
Published: (2025)

On the Impacts of Contexts on Repository-Level Code Generation
by: Hai, Nam Le, et al.
Published: (2024)

On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing
by: Kovrigin, Alexander, et al.
Published: (2024)

CATCODER: Repository-Level Code Generation with Relevant Code and Type Context
by: Pan, Zhiyuan, et al.
Published: (2024)

An Empirical Study of Proactive Coding Assistants in Real-World Software Development
by: Li, Lehui, et al.
Published: (2026)

REPOFUSE: Repository-Level Code Completion with Fused Dual Context
by: Liang, Ming, et al.
Published: (2024)

ATime-Consistent Benchmark for Repository-Level Software Engineering Evaluation
by: Xianpeng, et al.
Published: (2026)

SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding
by: Cai, Songcheng, et al.
Published: (2026)

Class-Level Code Generation from Natural Language Using Iterative, Tool-Enhanced Reasoning over Repository
by: Deshpande, Ajinkya, et al.
Published: (2024)

RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair
by: Pan, Zhongqiang, et al.
Published: (2026)

VulnAgent-X: A Layered Agentic Framework for Repository-Level Vulnerability Detection
by: Meng, Renwei, et al.
Published: (2026)

Needle in the Repo: A Benchmark for Maintainability in AI-Generated Repository Edits
by: Zhu, Haichao, et al.
Published: (2026)

Persistent Cross-Attempt State Optimization for Repository-Level Code Generation
by: Pan, Ruwei, et al.
Published: (2026)

In Line with Context: Repository-Level Code Generation via Context Inlining
by: Hu, Chao, et al.
Published: (2026)

ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
by: Hong, Jiseung, et al.
Published: (2026)

Toward Executable Repository-Level Code Generation via Environment Alignment
by: Pan, Ruwei, et al.
Published: (2026)

AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Context
by: Zhang, Lei, et al.
Published: (2026)

From Context to Intent: Reasoning-Guided Function-Level Code Completion
by: Li, Yanzhou, et al.
Published: (2025)

Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarks
by: Chu, Junjie, et al.
Published: (2026)

MemRepair: Hierarchical Memory for Agentic Repository-Level Vulnerability Repair
by: Liu, Simiao, et al.
Published: (2026)

Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
by: Gloaguen, Thibaud, et al.
Published: (2026)

AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
by: Jiang, Tianyue, et al.
Published: (2026)

Beyond Function-Level Search: Repository-Aware Dual-Encoder Code Retrieval with Adversarial Verification
by: Liu, Aofan, et al.
Published: (2025)

CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning
by: Lam, Man Ho, et al.
Published: (2025)

RepoHyper: Search-Expand-Refine on Semantic Graphs for Repository-Level Code Completion
by: Phan, Huy N., et al.
Published: (2024)

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
by: Ma, Jeffrey Jian, et al.
Published: (2025)

DecompileBench: A Comprehensive Benchmark for Evaluating Decompilers in Real-World Scenarios
by: Gao, Zeyu, et al.
Published: (2025)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
by: Duston, Titouan, et al.
Published: (2025)