Cover Image

DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hong, Sirui, Liu, Zhijie, Li, Tengfei, Tao, Wei, Wu, Yifan, Wu, Chenglin
Format:	Preprint
Published:	2026
Subjects:	Software Engineering Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluation
by: Bian, Yutong, et al.
Published: (2025)

EvalSVA: Multi-Agent Evaluators for Next-Gen Software Vulnerability Assessment
by: Wen, Xin-Cheng, et al.
Published: (2024)

AgentArcEval: An Architecture Evaluation Method for Foundation Model based Agents
by: Lu, Qinghua, et al.
Published: (2025)

DevEval: Evaluating Code Generation in Practical Software Projects
by: Li, Jia, et al.
Published: (2024)

From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
by: Ma, Murong, et al.
Published: (2026)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
by: Paul, Debalina Ghosh, et al.
Published: (2024)

Process-Level Trajectory Evaluation for Environment Configuration in Software Engineering Agents
by: Kuang, Jiayi, et al.
Published: (2025)

Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories
by: Bouzenia, Islem, et al.
Published: (2025)

RepoMasterEval: Evaluating Code Completion via Real-World Repositories
by: Wu, Qinyun, et al.
Published: (2024)

Benchmarks for Trajectory Safety Evaluation and Diagnosis in OpenClaw and Codex: ATBench-Claw and ATBench-Codex
by: Yang, Zhonghao, et al.
Published: (2026)

EvoClaw: Evaluating AI Agents on Continuous Software Evolution
by: Deng, Gangda, et al.
Published: (2026)

ProSoftArena: Benchmarking Hierarchical Capabilities of Multimodal Agents in Professional Software Environments
by: Ai, Jiaxin, et al.
Published: (2025)

MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering
by: Guo, Chuanzhe, et al.
Published: (2026)

TaskEval: Synthesised Evaluation for Foundation-Model Tasks
by: Widanapathiranage, Dilani, et al.
Published: (2025)

SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training
by: Wen, Xin-Cheng, et al.
Published: (2026)

GUITestScape: Towards Open-set Evaluation on Exploratory GUI Testing
by: Chen, Xiaoyi, et al.
Published: (2026)

Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows
by: Li, Chenxin, et al.
Published: (2026)

AndroidControl-Curated: Revealing the True Potential of GUI Agents through Benchmark Purification
by: Leung, Ho Fai, et al.
Published: (2025)

TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance
by: Bruches, Elena, et al.
Published: (2026)

Zero-Permission Manipulation: Can We Trust Large Multimodal Model Powered GUI Agents?
by: Qian, Yi, et al.
Published: (2026)

OSS-UAgent: An Agent-based Usability Evaluation Framework for Open Source Software
by: Meng, Lingkai, et al.
Published: (2025)

Unified Software Engineering Agent as AI Software Engineer
by: Applis, Leonhard, et al.
Published: (2025)

MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems
by: Jia, Jin, et al.
Published: (2026)

SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning
by: Chi, Yizhou, et al.
Published: (2024)

daVinci-Dev: Agent-native Mid-training for Software Engineering
by: Zeng, Ji, et al.
Published: (2026)

Self-Evolving Software Agents
by: Robol, Marco, et al.
Published: (2026)

Beyond Isolated Tasks: A Framework for Evaluating Coding Agents on Sequential Software Evolution
by: Shastry, KN Ajay, et al.
Published: (2026)

Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software
by: Ran, Dezhi, et al.
Published: (2024)

Taming System Complexity: Demystifying Software Engineering Agents in Diagnosing Linux Kernel Faults
by: Zhou, Zhenhao, et al.
Published: (2025)

Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
by: Trae Research Team, et al.
Published: (2025)

AgentStepper: Interactive Debugging of Software Development Agents
by: Hutter, Robert, et al.
Published: (2026)

OpenComputer: Verifiable Software Worlds for Computer-Use Agents
by: Wei, Jinbiao, et al.
Published: (2026)

Same Signal, Different Semantics: A Cross-Framework Behavioral Analysis of Software Engineering Agents
by: Ma, Wei, et al.
Published: (2026)

SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
by: Fan, Zhiyu, et al.
Published: (2025)

Towards Adaptive Software Agents for Debugging
by: Majdoub, Yacine, et al.
Published: (2025)

RoadmapBench: Evaluating Long-Horizon Agentic Software Development Across Version Upgrades
by: Xu, Xinbo, et al.
Published: (2026)

Coding with Eyes: Visual Feedback Unlocks Reliable GUI Code Generating and Debugging
by: Liu, Zhilin, et al.
Published: (2026)

AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?
by: Ran, Dezhi, et al.
Published: (2025)

AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
by: Zhang, Tanghaoran, et al.
Published: (2026)

Reducing Cost of LLM Agents with Trajectory Reduction
by: Xiao, Yuan-An, et al.
Published: (2025)