Cover Image

Evaluating Plan Compliance in Autonomous Programming Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Shuyang, Dehghan, Saman, Ganhotra, Jatin, Hirzel, Martin, Jabbarvand, Reyhaneh
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Process-Centric Analysis of Agentic Software Systems
by: Liu, Shuyang, et al.
Published: (2025)

CodeMind: Evaluating Large Language Models for Code Reasoning
by: Liu, Changshu, et al.
Published: (2024)

LeTI: Learning to Generate from Textual Interactions
by: Wang, Xingyao, et al.
Published: (2023)

Resolving Java Code Repository Issues with iSWE Agent
by: Ganhotra, Jatin, et al.
Published: (2026)

Translating Large-Scale C Repositories to Idiomatic Rust
by: Dehghan, Saman, et al.
Published: (2025)

SPARC: Scenario Planning and Reasoning for Automated C Unit Test Generation
by: Chowdhury, Jaid Monwar, et al.
Published: (2026)

Narrowing the Complexity Gap in the Evaluation of Large Language Models
by: Chen, Yang, et al.
Published: (2026)

Representing Prompting Patterns with PDL: Compliance Agent Case Study
by: Vaziri, Mandana, et al.
Published: (2025)

When Agents go Astray: Course-Correcting SWE Agents with PRMs
by: Gandhi, Shubham, et al.
Published: (2025)

Reproduction Test Generation for Java SWE Issues
by: Ahmed, Toufique, et al.
Published: (2026)

Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
by: Ahmed, Toufique, et al.
Published: (2025)

Can Old Tests Do New Tricks for Resolving SWE Issues?
by: Chen, Yang, et al.
Published: (2025)

Investigating Test Overfitting on SWE-bench
by: Ahmed, Toufique, et al.
Published: (2025)

Chain-of-Programming (CoP) : Empowering Large Language Models for Geospatial Code Generation
by: Hou, Shuyang, et al.
Published: (2024)

KAPSO: A Knowledge-grounded framework for Autonomous Program Synthesis and Optimization
by: Nadafian, Alireza, et al.
Published: (2026)

SWE-AGI: Benchmarking Specification-Driven Software Construction with MoonBit in the Era of Autonomous Agents
by: Zhang, Zhirui, et al.
Published: (2026)

A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models
by: Liu, Changshu, et al.
Published: (2025)

EvoConfig: Self-Evolving Multi-Agent Systems for Efficient Autonomous Environment Configuration
by: Guo, Xinshuai, et al.
Published: (2026)

CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging
by: Islam, Md. Ashraful, et al.
Published: (2025)

OmniCode: A Benchmark for Evaluating Software Engineering Agents
by: Sonwane, Atharv, et al.
Published: (2026)

A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning
by: Li, Xinzhe
Published: (2024)

Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
by: Kuang, Jiayi, et al.
Published: (2025)

A Generic Approach to Fix Test Flakiness in Real-World Projects
by: Chen, Yang, et al.
Published: (2024)

Otter: Generating Tests from Issues to Validate SWE Patches
by: Ahmed, Toufique, et al.
Published: (2025)

UA-Code-Bench: A Competitive Programming Benchmark for Evaluating LLM Code Generation in Ukrainian
by: Syromiatnikov, Mykyta, et al.
Published: (2025)

SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration
by: Chen, Jialong, et al.
Published: (2026)

CR-Bench: Evaluating the Real-World Utility of AI Code Review Agents
by: Pereira, Kristen, et al.
Published: (2026)

SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories
by: Bogin, Ben, et al.
Published: (2024)

Knowledge-Augmented Reasoning for EUAIA Compliance and Adversarial Robustness of LLMs
by: Momcilovic, Tomas Bueno, et al.
Published: (2024)

Planning to Explore: Curiosity-Driven Planning for LLM Test Generation
by: Amayuelas, Alfonso, et al.
Published: (2026)

CursorCore: Assist Programming through Aligning Anything
by: Jiang, Hao, et al.
Published: (2024)

RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation
by: Luo, Jane, et al.
Published: (2025)

CP-Agent: Agentic Constraint Programming
by: Szeider, Stefan
Published: (2025)

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks
by: Zhou, Xin, et al.
Published: (2025)

Evaluating Retrieval-Augmented Generation Variants for Natural Language-Based SQL and API Call Generation
by: Marketsmüller, Michael, et al.
Published: (2026)

Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents
by: Yang, Zonghan, et al.
Published: (2025)

Enabling Autonomic Microservice Management through Self-Learning Agents
by: Yu, Fenglin, et al.
Published: (2025)

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language
by: Cheng, Junhang, et al.
Published: (2026)

LocAgent: Graph-Guided LLM Agents for Code Localization
by: Chen, Zhaoling, et al.
Published: (2025)

Web Agents Should Adopt the Plan-Then-Execute Paradigm
by: Piet, Julien, et al.
Published: (2026)