Cover Image

HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Dewu, Wang, Yanlin, Shi, Ensheng, Zhang, Ruikai, Ma, Yuchi, Zhang, Hongyu, Zheng, Zibin
Format:	Preprint
Published:	2024
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
by: Zheng, Dewu, et al.
Published: (2024)

When to Stop? Towards Efficient Code Generation in LLMs with Excess Token Prevention
by: Guo, Lianghong, et al.
Published: (2024)

RLCoder: Reinforcement Learning for Repository-Level Code Completion
by: Wang, Yanlin, et al.
Published: (2024)

AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion
by: Jiang, Tianyue, et al.
Published: (2026)

A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends
by: Zheng, Zibin, et al.
Published: (2023)

Towards an Understanding of Context Utilization in Code Intelligence
by: Wang, Yanlin, et al.
Published: (2025)

SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA
by: Zhang, Jing, et al.
Published: (2025)

RustRepoTrans: Repository-level Code Translation Benchmark Targeting Rust
by: Ou, Guangsheng, et al.
Published: (2024)

EffiReasonTrans: RL-Optimized Reasoning for Code Translation
by: Wang, Yanlin, et al.
Published: (2025)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context Poisoning
by: Wang, Yanlin, et al.
Published: (2026)

RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
by: Liang, Linxi, et al.
Published: (2025)

RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
by: Wang, Yanlin, et al.
Published: (2026)

SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization
by: Wang, Yanlin, et al.
Published: (2024)

You Augment Me: Exploring ChatGPT-based Data Augmentation for Semantic Code Search
by: Wang, Yanlin, et al.
Published: (2024)

GitEvo: Code Evolution Analysis for Git Repositories
by: Hora, Andre
Published: (2026)

OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution
by: Guo, Lianghong, et al.
Published: (2025)

ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation
by: Liu, Sicong, et al.
Published: (2026)

Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition
by: Liu, Mingwei, et al.
Published: (2026)

SECRET: Towards Scalable and Efficient Code Retrieval via Segmented Deep Hashing
by: Gu, Wenchao, et al.
Published: (2024)

What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond
by: Gu, Wenchao, et al.
Published: (2025)

Agents in Software Engineering: Survey, Landscape, and Vision
by: Wang, Yanlin, et al.
Published: (2024)

A Hierarchical and Evolvable Benchmark for Fine-Grained Code Instruction Following with Multi-Turn Feedback
by: Duan, Guoliang, et al.
Published: (2025)

Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
by: Wang, Yanlin, et al.
Published: (2024)

FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
by: Dai, Dekun, et al.
Published: (2025)

Evolving Triple Knowledge-Augmented LLMs for Code Translation in Repository Context
by: Ou, Guangsheng, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

KTester: Leveraging Domain and Testing Knowledge for More Effective LLM-based Test Generation
by: Li, Anji, et al.
Published: (2025)

Trace: Securing Smart Contract Repository Against Access Control Vulnerability
by: Chen, Chong, et al.
Published: (2025)

LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation
by: Zhang, Ziyao, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

Efficiently Detecting Reentrancy Vulnerabilities in Complex Smart Contracts
by: Wang, Zexu, et al.
Published: (2024)

A Preliminary Study on the Robustness of Code Generation by Large Language Models
by: Li, Zike, et al.
Published: (2025)

Identifying Smart Contract Security Issues in Code Snippets from Stack Overflow
by: Chen, Jiachi, et al.
Published: (2024)

CodeFuse-CR-Bench: A Comprehensiveness-aware Benchmark for End-to-End Code Review Evaluation in Python Projects
by: Guo, Hanyang, et al.
Published: (2025)

AnomalyGen: Enhancing Log-Based Anomaly Detection with Code-Guided Data Augmentation
by: Li, Xinyu, et al.
Published: (2026)

SolEval: Benchmarking Large Language Models for Repository-level Solidity Code Generation
by: Peng, Zhiyuan, et al.
Published: (2025)

Dynamic analysis enhances issue resolution
by: Liu, Mingwei, et al.
Published: (2026)

Benchmarking and Studying the LLM-based Code Review
by: Zeng, Zhengran, et al.
Published: (2025)

EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
by: Zhang, Wentao, et al.
Published: (2026)