Cover Image

GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ni, Ziyi, Wang, Huacan, Zhang, Shuo, Lu, Shuo, He, Ziyang, You, Wang, Tang, Zhenheng, Du, Yuntao, Sun, Bill, Liu, Hongzhang, Hu, Sen, Chen, Ronghao, Li, Bo, Li, Xin, Hu, Chen, Jiao, Binxing, Jiang, Daxin, Lyu, Pin
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving
by: Wang, Huacan, et al.
Published: (2025)

SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents
by: Lin, Jiaye, et al.
Published: (2025)

CloneMem: Benchmarking Long-Term Memory for AI Clones
by: Hu, Sen, et al.
Published: (2026)

KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions
by: Wu, Tingyu, et al.
Published: (2026)

Controlled Self-Evolution for Algorithmic Code Optimization
by: Hu, Tu, et al.
Published: (2026)

SkillGenBench: Benchmarking Skill Generation Pipelines for LLM Agents
by: Zhou, Yifan, et al.
Published: (2026)

UIPress: Bringing Optical Token Compression to UI-to-Code Generation
by: Dai, Dasen, et al.
Published: (2026)

GitEvo: Code Evolution Analysis for Git Repositories
by: Hora, Andre
Published: (2026)

CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks
by: Fu, Lingyue, et al.
Published: (2025)

MemGovern: Enhancing Code Agents through Learning from Governed Human Experiences
by: Wang, Qihao, et al.
Published: (2026)

ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
by: Tang, Xiangru, et al.
Published: (2023)

Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration
by: Guo, Yifu, et al.
Published: (2025)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
by: Duston, Titouan, et al.
Published: (2025)

EvoFSM: Controllable Self-Evolution for Deep Research with Finite State Machines
by: Zhang, Shuo, et al.
Published: (2026)

Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handling
by: Ni, Ziyi, et al.
Published: (2024)

SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
by: Li, Kuan, et al.
Published: (2026)

Sema Code: Decoupling AI Coding Agents into Programmable, Embeddable Infrastructure
by: Wang, Huacan, et al.
Published: (2026)

CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
by: Jiang, Hongchao, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

Automated Benchmark Generation for Repository-Level Coding Tasks
by: Vergopoulos, Konstantinos, et al.
Published: (2025)

A Benchmark for Evaluating Repository-Level Code Agents with Intermediate Reasoning on Feature Addition Task
by: Liu, Shuhan, et al.
Published: (2026)

Sandboxed Coding Agents are Competitive Omni-modal Task Solvers
by: Chen, Dongping, et al.
Published: (2026)

Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis
by: Lin, Ronghao, et al.
Published: (2022)

Uncovering Code Insights: Leveraging GitHub Artifacts for Deeper Code Understanding
by: Nevo, Ziv, et al.
Published: (2025)

Improving Code Localization with Repository Memory
by: Wang, Boshi, et al.
Published: (2025)

SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
by: Shen, Chihao, et al.
Published: (2025)

MRG-Bench: Evaluating and Exploring the Requirements of Context for Repository-Level Code Generation
by: Li, Haiyang
Published: (2025)

Tree-of-Code: A Hybrid Approach for Robust Complex Task Planning and Execution
by: Ni, Ziyi, et al.
Published: (2024)

EpochX: Building the Infrastructure for an Emergent Agent Civilization
by: Wang, Huacan, et al.
Published: (2026)

HomeFlow: A Data Flywheel for Smart Home Agent Training with Verifiable Simulation
by: Gu, Yi, et al.
Published: (2026)

Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory
by: Hu, Sen, et al.
Published: (2026)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

CodeS: Natural Language to Code Repository via Multi-Layer Sketch
by: Zan, Daoguang, et al.
Published: (2024)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

CATCODER: Repository-Level Code Generation with Relevant Code and Type Context
by: Pan, Zhiyuan, et al.
Published: (2024)

MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion benchmark
by: Zou, Qingyun, et al.
Published: (2026)

Co-Evolution of Types and Dependencies: Towards Repository-Level Type Inference for Python Code
by: Sun, Shuo, et al.
Published: (2025)

AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Context
by: Zhang, Lei, et al.
Published: (2026)

Immersion in the GitHub Universe: Scaling Coding Agents to Mastery
by: Zhao, Jiale, et al.
Published: (2026)

TypeScript Repository Indexing for Code Agent Retrieval
by: Pu, Junsong, et al.
Published: (2026)