Cover Image

HardTests: Synthesizing High-Quality Test Cases for LLM Coding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Zhongmou, Choi, Yee Man, Zhang, Kexun, Ji, Jiabao, Zhou, Junting, Xu, Dejia, Bercovich, Ivan, Zhang, Aidan, Li, Lei
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions
by: Zhang, Kexun, et al.
Published: (2024)

Terminal Wrench: A Dataset of 331 Reward-Hackable Environments and 3,632 Exploit Trajectories
by: Bercovich, Ivan, et al.
Published: (2026)

What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design
by: Bercovich, Ivan
Published: (2026)

Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learning
by: Fu, Jia, et al.
Published: (2025)

HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning
by: Liu, Yujian, et al.
Published: (2025)

CodeContests+: High-Quality Test Case Generation for Competitive Programming
by: Wang, Zihan, et al.
Published: (2025)

Quantum-Guided Test Case Minimization for LLM-Based Code Generation
by: Zhang, Huixiang, et al.
Published: (2025)

Scaling LLM Inference with Optimized Sample Compute Allocation
by: Zhang, Kexun, et al.
Published: (2024)

Synthesizing Test Cases for Narrowing Specification Candidates
by: Cunha, Alcino, et al.
Published: (2025)

Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputs
by: Ji, Zhenlan, et al.
Published: (2024)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
by: Zhao, Songwen, et al.
Published: (2025)

Test-time Correlation Alignment
by: You, Linjing, et al.
Published: (2025)

Rethinking Verification for LLM Code Generation: From Generation to Testing
by: Ma, Zihan, et al.
Published: (2025)

Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLM
by: Yu, Jiawei, et al.
Published: (2024)

On the Hardness of Conditional Independence Testing In Practice
by: He, Zheng, et al.
Published: (2025)

ABFS: Natural Robustness Testing for LLM-based NLP Software
by: Xiao, Mingxuan, et al.
Published: (2025)

Bias Testing and Mitigation in LLM-based Code Generation
by: Huang, Dong, et al.
Published: (2023)

Testing Sumsets is Hard
by: Chen, Xi, et al.
Published: (2024)

AutoTest: Evolutionary Code Solution Selection with Test Cases
by: Duan, Zhihua, et al.
Published: (2024)

SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner
by: Zhang, Lei, et al.
Published: (2025)

Self-Testing Quantum Error Correcting Codes: Analyzing Computational Hardness
by: Kuo, En-Jui, et al.
Published: (2024)

CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
by: Zhu, Xiao, et al.
Published: (2026)

ERASOR++: Height Coding Plus Egocentric Ratio Based Dynamic Object Removal for Static Point Cloud Mapping
by: Zhang, Jiabao, et al.
Published: (2024)

Assessing the Robustness of LLM-based NLP Software via Automated Testing
by: Xiao, Mingxuan, et al.
Published: (2024)

LLM-Based Test Case Generation in DBMS through Monte Carlo Tree Search
by: Chen, Yujia, et al.
Published: (2026)

Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learning
by: Wang, Aozhe, et al.
Published: (2026)

Dynamic Test Case Prioritization in Industrial Test Result Datasets
by: Torbunova, Alina, et al.
Published: (2024)

A Bayes-Motivated Quadratic-Form Test for High-Dimensional Mean Testing
by: He, Daojiang, et al.
Published: (2025)

Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decoding
by: Zhang, Kexun, et al.
Published: (2023)

Ising-based Test Optimization and Benchmarking
by: Yang, Yige, et al.
Published: (2026)

Test Set Quality in Multilingual LLM Evaluation
by: Kranti, Chalamalasetti, et al.
Published: (2025)

Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation
by: Cui, Yi
Published: (2025)

ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code Generation
by: Sun, Hui, et al.
Published: (2026)

Revisit Self-Debugging with Self-Generated Tests for Code Generation
by: Chen, Xiancai, et al.
Published: (2025)

DAJ: Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation
by: Qin, Peijia, et al.
Published: (2026)

Identifying Flaky Tests in Quantum Code: A Machine Learning Approach
by: Kaur, Khushdeep, et al.
Published: (2025)

Can LLMs Generate High-Quality Test Cases for Algorithm Problems? TestCase-Eval: A Systematic Evaluation of Fault Coverage and Exposure
by: Yang, Zheyuan, et al.
Published: (2025)

FRET: Feature Redundancy Elimination for Test Time Adaptation
by: You, Linjing, et al.
Published: (2025)

Responsible AI for Test Equity and Quality: The Duolingo English Test as a Case Study
by: Burstein, Jill, et al.
Published: (2024)

Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction
by: Li, Xiang, et al.
Published: (2026)