Cover Image

ContractEval: A Benchmark for Evaluating Contract-Satisfying Assertions in Code Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lim, Soohan, Hahn, Joonghyuk, Park, Hyunwoo, Ko, Sang-Ki, Han, Yo-Sub
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Software Engineering 68T01 I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MEC$^3$O: Multi-Expert Consensus for Code Time Complexity Prediction
by: Hahn, Joonghyuk, et al.
Published: (2025)

TCProF: Time-Complexity Prediction SSL Framework
by: Hahn, Joonghyuk, et al.
Published: (2025)

Repairing Regex Vulnerabilities via Localization-Guided Instructions
by: Sung, Sicheol, et al.
Published: (2025)

Benchmarking Energy Efficiency of Large Language Models Using vLLM
by: Pronk, K., et al.
Published: (2025)

AmpleHate: Amplifying the Attention for Versatile Implicit Hate Detection
by: Lee, Yejin, et al.
Published: (2025)

Mind the Metrics: Patterns for Telemetry-Aware In-IDE AI Application Development using the Model Context Protocol (MCP)
by: Koc, Vincent, et al.
Published: (2025)

Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models
by: Lee, Yejin, et al.
Published: (2026)

A transfer learning approach for automatic conflicts detection in software requirement sentence pairs based on dual encoders
by: Wang, Yizheng, et al.
Published: (2025)

SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions
by: Xu, Weijie, et al.
Published: (2025)

5C Prompt Contracts: A Minimalist, Creative-Friendly, Token-Efficient Design Framework for Individual and SME LLM Usage
by: Ari, Ugur
Published: (2025)

AI-assisted German Employment Contract Review: A Benchmark Dataset
by: Wardas, Oliver, et al.
Published: (2025)

ContractBench: Can LLM Agents Preserve Observation Contracts?
by: Wang, Jicheng, et al.
Published: (2026)

An Epidemiological Knowledge Graph extracted from the World Health Organization's Disease Outbreak News
by: Consoli, Sergio, et al.
Published: (2025)

SLEAN: Simple Lightweight Ensemble Analysis Network for Multi-Provider LLM Coordination: Design, Implementation, and Vibe Coding Bug Investigation Case Study
by: Vargas, Matheus J. T.
Published: (2025)

RV-HATE: Reinforced Multi-Module Voting for Implicit Hate Speech Detection
by: Lee, Yejin, et al.
Published: (2025)

MicroRemed: Benchmarking LLMs in Microservices Remediation
by: Zhang, Lingzhe, et al.
Published: (2025)

LLMs for Legal Subsumption in German Employment Contracts
by: Wardas, Oliver, et al.
Published: (2025)

Simple and Effective Baselines for Code Summarisation Evaluation
by: Robinson, Jade, et al.
Published: (2025)

Tool-Genesis: A Task-Driven Tool Creation Benchmark for Self-Evolving Language Agent
by: Xia, Bowei, et al.
Published: (2026)

Large Language Models are Inconsistent and Biased Evaluators
by: Stureborg, Rickard, et al.
Published: (2024)

AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems
by: Yagoubi, Faouzi El, et al.
Published: (2026)

A Comparative Study of Feature Selection in Tsetlin Machines
by: Halenka, Vojtech, et al.
Published: (2025)

Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo
by: Ekle, Ocheme Anthony, et al.
Published: (2025)

Solving Zebra Puzzles Using Constraint-Guided Multi-Agent Systems
by: Berman, Shmuel, et al.
Published: (2024)

From MTEB to MTOB: Retrieval-Augmented Classification for Descriptive Grammars
by: Kornilov, Albert, et al.
Published: (2024)

BEATS: Bias Evaluation and Assessment Test Suite for Large Language Models
by: Abhishek, Alok, et al.
Published: (2025)

Reasoning aligns language models to human cognition
by: Guiomar, Gonçalo, et al.
Published: (2026)

Project Synapse: A Hierarchical Multi-Agent Framework with Hybrid Memory for Autonomous Resolution of Last-Mile Delivery Disruptions
by: Yadav, Arin Gopalan, et al.
Published: (2026)

Benchmarking Deception Probes via Black-to-White Performance Boosts
by: Parrack, Avi, et al.
Published: (2025)

Controlling Long-Horizon Behavior in Language Model Agents with Explicit State Dynamics
by: Subaharan, Sukesh
Published: (2026)

When Many-Shot Prompting Fails: An Empirical Study of LLM Code Translation
by: Oskooei, Amirkia Rafiei, et al.
Published: (2025)

QHackBench: Benchmarking Large Language Models for Quantum Code Generation Using PennyLane Hackathon Challenges
by: Basit, Abdul, et al.
Published: (2025)

Commenting Higher-level Code Unit: Full Code, Reduced Code, or Hierarchical Code Summarization
by: Sun, Weisong, et al.
Published: (2025)

STAB: Specification-driven Testing for Algorithmic Bottlenecks
by: Lim, Soohan, et al.
Published: (2026)

Feature-Factory: Automating Software Feature Integration Using Generative AI
by: Vsevolodovna, Ruslan Idelfonso Magana
Published: (2024)

CWM: An Open-Weights LLM for Research on Code Generation with World Models
by: FAIR CodeGen team, et al.
Published: (2025)

Vibe-Creation: The Epistemology of Human-AI Emergent Cognition
by: Levin, Ilya
Published: (2026)

Navigational Thinking as an Emerging Paradigm of Computer Science in the Age of Generative AI
by: Levin, Ilya
Published: (2026)

ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization
by: Fang, Chunrong, et al.
Published: (2024)

HumanMCP: A Human-Like Query Dataset for Evaluating MCP Tool Retrieval Performance
by: Laddha, Shubh, et al.
Published: (2025)