Cover Image

ContractBench: Can LLM Agents Preserve Observation Contracts?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jicheng, He, Yifeng, Wang, Zili, Xing, Hanwen, De, Arkaprava, Chen, Hao
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence I.2.7; I.2.6
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Automated Bug Triaging using Instruction-Tuned Large Language Models
by: Kiashemshaki, Kiana, et al.
Published: (2025)

5C Prompt Contracts: A Minimalist, Creative-Friendly, Token-Efficient Design Framework for Individual and SME LLM Usage
by: Ari, Ugur
Published: (2025)

AuditRepairBench: A Paired-Execution Trace Corpus for Evaluator-Channel Ranking Instability in Agent Repair
by: Hu, Yuelin, et al.
Published: (2026)

Toward Architecture-Aware Evaluation Metrics for LLM Agents
by: Souza, Débora, et al.
Published: (2026)

CIDR: A Large-Scale Industrial Source Code Dataset for Software Engineering Research
by: Savenkov, Vladislav
Published: (2026)

AdaDec: A Uncertainty-Guided Lookahead Decoding Framework for LLM-Based Code Generation
by: He, Kaifeng, et al.
Published: (2025)

PARNESS: A Paper Harness for End-to-End Automated Scientific Research with Dynamic Workflows, Full-Text Indexing, and Cross-Run Knowledge Accumulation
by: Wang, Yuchen, et al.
Published: (2026)

AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
by: Mazaheri, Parsa, et al.
Published: (2026)

The Single-File Test: A Longitudinal Public-Interface Evaluation of First-Output LLM Web Generation with Social Reach Tracking
by: Palacios, Diego Cabezas
Published: (2026)

FORGE: An LLM-driven Framework for Large-Scale Smart Contract Vulnerability Dataset Construction
by: Chen, Jiachi, et al.
Published: (2025)

Can AI Assist in Olympiad Coding
by: Ren, Samuel
Published: (2025)

MOCHA: Multi-Objective Chebyshev Annealing for Agent Skill Optimization
by: Tanjim, Md Mehrab, et al.
Published: (2026)

SiliconMind-V1: Multi-Agent Distillation and Debug-Reasoning Workflows for Verilog Code Generation
by: Chen, Mu-Chi, et al.
Published: (2026)

Tool-Genesis: A Task-Driven Tool Creation Benchmark for Self-Evolving Language Agent
by: Xia, Bowei, et al.
Published: (2026)

ContractEval: A Benchmark for Evaluating Contract-Satisfying Assertions in Code Generation
by: Lim, Soohan, et al.
Published: (2025)

Software Defined Vehicle Code Generation: A Few-Shot Prompting Approach
by: Nguyen, Quang-Dung, et al.
Published: (2025)

AgentPulse: A Continuous Multi-Signal Framework for Evaluating AI Agents in Deployment
by: Gao, Yuxuan, et al.
Published: (2026)

LLMCup: Ranking-Enhanced Comment Updating with LLMs
by: Ge, Hua, et al.
Published: (2025)

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
by: Agrawal, Lakshya A, et al.
Published: (2025)

EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization
by: Zhang, Jiahao, et al.
Published: (2026)

Predictive Analytics for Collaborators Answers, Code Quality, and Dropout on Stack Overflow
by: Zolduoarrati, Elijah, et al.
Published: (2025)

TRACE: A taxonomy-grounded synthetic dataset for teaching-program generation and session interpretation in Applied Behavior Analysis
by: Kahunla, Festus
Published: (2026)

Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
by: Cartagena, Arnold, et al.
Published: (2026)

Merge-Bench: Resolve Merge Conflicts with Large Language Models
by: Schesch, Benedikt, et al.
Published: (2026)

Generative AI Toolkit -- a framework for increasing the quality of LLM-based applications over their whole life cycle
by: Kohl, Jens, et al.
Published: (2024)

When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges
by: Darshan, Parth, et al.
Published: (2026)

Collaborative LLM Agents for C4 Software Architecture Design Automation
by: Szczepanik, Kamil, et al.
Published: (2025)

Human-in-the-Loop Benchmarking of Heterogeneous LLMs for Automated Competency Assessment in Secondary Level Mathematics
by: Bhusal, Jatin, et al.
Published: (2026)

Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automation
by: Trooskens, Geert, et al.
Published: (2026)

Feedback-Normalized Developer Memory for Reinforcement-Learning Coding Agents: A Safety-Gated MCP Architecture
by: Iscan, Mehmet
Published: (2026)

A Framework for Testing and Adapting REST APIs as LLM Tools
by: Bandlamudi, Jayachandu, et al.
Published: (2025)

Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation
by: Young, Richard J.
Published: (2025)

AgentModernize: Preserving Business Logic in Legacy Modernization with Multi-Agent LLMs and Behavioral Specification Graphs
by: Ahmed, Sheikh Nazib, et al.
Published: (2026)

Learning Software Bug Reports: A Systematic Literature Review
by: Long, Guoming, et al.
Published: (2025)

RelRepair: Enhancing Automated Program Repair by Retrieving Relevant Code
by: Liu, Shunyu, et al.
Published: (2025)

Contrastive Learning-Enhanced Large Language Models for Monolith-to-Microservice Decomposition
by: Sellami, Khaled, et al.
Published: (2025)

FREYR: A Framework for Recognizing and Executing Your Requests
by: Gallotta, Roberto, et al.
Published: (2025)

Comprehensive Evaluation and Insights into the Use of Large Language Models in the Automation of Behavior-Driven Development Acceptance Test Formulation
by: Karpurapu, Shanthi, et al.
Published: (2024)

Towards Observation Lakehouses: Living, Interactive Archives of Software Behavior
by: Kessel, Marcus
Published: (2025)

Engineering A Large Language Model From Scratch
by: Oketunji, Abiodun Finbarrs
Published: (2024)