Cover Image

Benchmarking LLMs for Unit Test Generation from Real-World Functions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Dong, Zhang, Jie M., Harman, Mark, Zhang, Qianru, Du, Mingzhe, Ng, See-Kiong
Format:	Preprint
Published:	2025
Subjects:	Software Engineering Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Nexus: Execution-Grounded Multi-Agent Test Oracle Synthesis
by: Huang, Dong, et al.
Published: (2025)

Measuring the Influence of Incorrect Code on Test Generation
by: Huang, Dong, et al.
Published: (2024)

Mercury: A Code Efficiency Benchmark for Code Large Language Models
by: Du, Mingzhe, et al.
Published: (2024)

TestExplora: Benchmarking LLMs for Proactive Bug Discovery via Repository-Level Test Generation
by: Liu, Steven, et al.
Published: (2026)

YATE: The Role of Test Repair in LLM-Based Unit Test Generation
by: Konstantinou, Michael, et al.
Published: (2025)

DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Pattern
by: Yang, Lekang, et al.
Published: (2025)

EffiBench: Benchmarking the Efficiency of Automatically Generated Code
by: Huang, Dong, et al.
Published: (2024)

MultiFileTest: A Multi-File-Level LLM Unit Test Generation Benchmark and Impact of Error Fixing Mechanisms
by: Wang, Yibo, et al.
Published: (2025)

Dynamic Scaling of Unit Tests for Code Reward Modeling
by: Ma, Zeyao, et al.
Published: (2025)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
by: Zhao, Songwen, et al.
Published: (2025)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation
by: Ma, Zeyao, et al.
Published: (2024)

CodeArena: A Collective Evaluation Platform for LLM Code Generation
by: Du, Mingzhe, et al.
Published: (2025)

DependEval: Benchmarking LLMs for Repository Dependency Understanding
by: Du, Junjia, et al.
Published: (2025)

UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance
by: Ma, Yichuan, et al.
Published: (2025)

FairCoder: Evaluating Social Bias of LLMs in Code Generation
by: Du, Yongkang, et al.
Published: (2025)

ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
by: Yang, Jie, et al.
Published: (2026)

TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark
by: Jain, Kush, et al.
Published: (2024)

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations
by: Li, Jia, et al.
Published: (2024)

CONCUR: Benchmarking LLMs for Concurrent Code Generation
by: Huang, Jue, et al.
Published: (2026)

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation
by: Chen, Zaoyu, et al.
Published: (2026)

EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization
by: Huang, Dong, et al.
Published: (2024)

Can LLMs Generate Reliable Test Case Generators? A Study on Competition-Level Programming Problems
by: Cao, Yuhan, et al.
Published: (2025)

SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle
by: Zhao, Fufangchen, et al.
Published: (2024)

GUI Test Migration via Abstraction and Concretization
by: Zhang, Yakun, et al.
Published: (2024)

EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning
by: Huang, Dong, et al.
Published: (2024)

Can LLMs Generate High-Quality Test Cases for Algorithm Problems? TestCase-Eval: A Systematic Evaluation of Fault Coverage and Exposure
by: Yang, Zheyuan, et al.
Published: (2025)

RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
by: Wang, Yanli, et al.
Published: (2024)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

RITFIS: Robust input testing framework for LLMs-based intelligent software
by: Xiao, Mingxuan, et al.
Published: (2024)

Evaluating LLMs on Sequential API Call Through Automated Test Generation
by: Huang, Yuheng, et al.
Published: (2025)

StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
by: Yang, Jialin, et al.
Published: (2025)

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language
by: Cheng, Junhang, et al.
Published: (2026)

PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization
by: Jing, Huihao, et al.
Published: (2026)

HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning
by: Liu, Yujian, et al.
Published: (2025)

A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
by: Guo, Jiale, et al.
Published: (2025)

ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
by: Liu, Kaiyuan, et al.
Published: (2025)

Learning to Generate Unit Tests for Automated Debugging
by: Prasad, Archiki, et al.
Published: (2025)

Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations
by: Salimian, Sina, et al.
Published: (2025)