Copertina

PBT-Bench: Benchmarking AI Agents on Property-Based Testing

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Jing, Lucas, Wang, Xinqi, Zhang, Liao, Du, Simon S.
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Software Engineering Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
di: Lu, Pengrui, et al.
Pubblicazione: (2026)

ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution
di: Wang, Yubang, et al.
Pubblicazione: (2026)

LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
di: Qiu, Jielin, et al.
Pubblicazione: (2025)

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
di: Merrill, Mike A., et al.
Pubblicazione: (2026)

GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging
di: Ni, Ziyi, et al.
Pubblicazione: (2025)

ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agents
di: Shen, Haiyang, et al.
Pubblicazione: (2024)

DataGovBench: Benchmarking LLM Agents for Real-World Data Governance Workflows
di: Liu, Zhou, et al.
Pubblicazione: (2025)

Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation
di: Garg, Spandan, et al.
Pubblicazione: (2025)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
di: Duston, Titouan, et al.
Pubblicazione: (2025)

UnitTenX: Generating Tests for Legacy Packages with AI Agents Powered by Formal Verification
di: Charalambous, Yiannis, et al.
Pubblicazione: (2025)

FeatureBench: Benchmarking Agentic Coding for Complex Feature Development
di: Zhou, Qixing, et al.
Pubblicazione: (2026)

DecompileBench: A Comprehensive Benchmark for Evaluating Decompilers in Real-World Scenarios
di: Gao, Zeyu, et al.
Pubblicazione: (2025)

ML-Dev-Bench: Comparative Analysis of AI Agents on ML development workflows
di: Padigela, Harshith, et al.
Pubblicazione: (2025)

BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks
di: Tu, Xinming, et al.
Pubblicazione: (2026)

Neuro-Symbolic Generation and Validation of Memory-Aware Formal Function Specifications
di: Zhang, Liao, et al.
Pubblicazione: (2026)

PyBench: Evaluating LLM Agent on various real-world coding tasks
di: Zhang, Yaolun, et al.
Pubblicazione: (2024)

RESTestBench: A Benchmark for Evaluating the Effectiveness of LLM-Generated REST API Test Cases from NL Requirements
di: Kogler, Leon, et al.
Pubblicazione: (2026)

CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories
di: Xiao, Yijia, et al.
Pubblicazione: (2025)

Code2Bench: Scaling Source and Rigor for Dynamic Benchmark Construction
di: Zhang, Zhe, et al.
Pubblicazione: (2025)

CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
di: Kim, Myeongsoo, et al.
Pubblicazione: (2025)

From Translation to Superset: Benchmark-Driven Evolution of a Production AI Agent from Rust to Python
di: Wang, Jinhua, et al.
Pubblicazione: (2026)

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
di: Zhang, Zehua, et al.
Pubblicazione: (2025)

EvoCodeBench: A Human-Performance Benchmark for Self-Evolving LLM-Driven Coding Systems
di: Zhang, Wentao, et al.
Pubblicazione: (2026)

FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation
di: Zhu, Hongda, et al.
Pubblicazione: (2025)

FVSpec: Real-World Property-Based Tests as Lean Challenges
di: Dougherty, Quinn, et al.
Pubblicazione: (2026)

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
di: Chen, Zhi, et al.
Pubblicazione: (2026)

SWE Context Bench: A Benchmark for Context Learning in Coding
di: Zhu, Jiayuan, et al.
Pubblicazione: (2026)

WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics
di: Liu, Chenxu, et al.
Pubblicazione: (2026)

CrackMeBench: Binary Reverse Engineering for Agents
di: David, Isaac, et al.
Pubblicazione: (2026)

FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
di: May, Victor, et al.
Pubblicazione: (2025)

Code Review Agent Benchmark
di: Zhang, Yuntong, et al.
Pubblicazione: (2026)

DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation
di: Xiao, Jingyu, et al.
Pubblicazione: (2025)

Agentic Property-Based Testing: Finding Bugs Across the Python Ecosystem
di: Maaz, Muhammad, et al.
Pubblicazione: (2025)

CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
di: Yin, Wenjing, et al.
Pubblicazione: (2025)

LoCoBench: A Benchmark for Long-Context Large Language Models in Complex Software Engineering
di: Qiu, Jielin, et al.
Pubblicazione: (2025)

WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing
di: Kong, Fanheng, et al.
Pubblicazione: (2026)

AXIOM: Benchmarking LLM-as-a-Judge for Code via Rule-Based Perturbation and Multisource Quality Calibration
di: Wang, Ruiqi, et al.
Pubblicazione: (2025)

SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents
di: Mündler, Niels, et al.
Pubblicazione: (2024)

GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git
di: Lindenbauer, Tobias, et al.
Pubblicazione: (2025)

SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation
di: He, Yibo, et al.
Pubblicazione: (2025)