Copertina

Stop Comparing LLM Agents Without Disclosing the Harness

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Yunbei, Wang, Janet, Ge, Yingqiang, Xu, Weijie, Hamm, Jihun, Reddy, Chandan K.
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Artificial Intelligence Software Engineering
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

PatchFinder: A Two-Phase Approach to Security Patch Tracing for Disclosed Vulnerabilities in Open-Source Software
di: Li, Kaixuan, et al.
Pubblicazione: (2024)

Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming
di: Agarwal, Anisha, et al.
Pubblicazione: (2024)

Reliable Graph-RAG for Codebases: AST-Derived Graphs vs LLM-Extracted Knowledge Graphs
di: Chinthareddy, Manideep Reddy
Pubblicazione: (2026)

REMODEL-LLM: Transforming C code to Java using LLMs
di: Gupta, Aryan, et al.
Pubblicazione: (2025)

Adaptive Root Cause Localization for Microservice Systems with Multi-Agent Recursion-of-Thought
di: Zhang, Lingzhe, et al.
Pubblicazione: (2025)

AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents
di: Zhong, Hailin, et al.
Pubblicazione: (2026)

Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvement
di: Liu, Fengrui, et al.
Pubblicazione: (2026)

Architecture Without Architects: How AI Coding Agents Shape Software Architecture
di: Konrad, Phongsakon Mark, et al.
Pubblicazione: (2026)

RFCAudit: An LLM Agent for Functional Bug Detection in Network Protocols
di: Zheng, Mingwei, et al.
Pubblicazione: (2025)

Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation
di: Zhang, Lingzhe, et al.
Pubblicazione: (2026)

Agents in the Wild: Safety, Society, and the Illusion of Sociality on Moltbook
di: Zhang, Yunbei, et al.
Pubblicazione: (2026)

MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
di: Tao, Wei, et al.
Pubblicazione: (2024)

Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planning
di: Zhang, Yunbei, et al.
Pubblicazione: (2026)

PyBench: Evaluating LLM Agent on various real-world coding tasks
di: Zhang, Yaolun, et al.
Pubblicazione: (2024)

Evolving Excellence: Automated Optimization of LLM-based Agents
di: Brookes, Paul, et al.
Pubblicazione: (2025)

Agentic Harness for Real-World Compilers
di: Zheng, Yingwei, et al.
Pubblicazione: (2026)

Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
di: Trae Research Team, et al.
Pubblicazione: (2025)

LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering
di: Qiu, Jielin, et al.
Pubblicazione: (2025)

LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Research
di: Yan, Shuo, et al.
Pubblicazione: (2025)

iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocols
di: Sun, Xikai, et al.
Pubblicazione: (2025)

RepairAgent: An Autonomous, LLM-Based Agent for Program Repair
di: Bouzenia, Islem, et al.
Pubblicazione: (2024)

ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety
di: Wang, Haoyu, et al.
Pubblicazione: (2025)

Z-Space: A Multi-Agent Tool Orchestration Framework for Enterprise-Grade LLM Automation
di: He, Qingsong, et al.
Pubblicazione: (2025)

From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents
di: Ran, Dezhi, et al.
Pubblicazione: (2025)

ClawSafety: "Safe" LLMs, Unsafe Agents
di: Wei, Bowen, et al.
Pubblicazione: (2026)

OrcaLoca: An LLM Agent Framework for Software Issue Localization
di: Yu, Zhongming, et al.
Pubblicazione: (2025)

EvoDev: An Iterative Feature-Driven Framework for End-to-End Software Development with LLM-based Agents
di: Liu, Junwei, et al.
Pubblicazione: (2025)

DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
di: Ma, Ming, et al.
Pubblicazione: (2025)

Reducing Cost of LLM Agents with Trajectory Reduction
di: Xiao, Yuan-An, et al.
Pubblicazione: (2025)

LLM Collaboration With Multi-Agent Reinforcement Learning
di: Liu, Shuo, et al.
Pubblicazione: (2025)

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents
di: Chen, Zhi, et al.
Pubblicazione: (2026)

From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
di: Wang, Yawen, et al.
Pubblicazione: (2026)

Effective Harness Engineering for Algorithm Discovery with Coding Agents
di: Ishibashi, Yoichi, et al.
Pubblicazione: (2026)

AI2Apps: A Visual IDE for Building LLM-based AI Agent Applications
di: Pang, Xin, et al.
Pubblicazione: (2024)

Harnessing the Power of LLMs: Automating Unit Test Generation for High-Performance Computing
di: Karanjai, Rabimba, et al.
Pubblicazione: (2024)

Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agents
di: Liu, Xiang, et al.
Pubblicazione: (2026)

AgentForge: Execution-Grounded Multi-Agent LLM Framework for Autonomous Software Engineering
di: Kumar, Rajesh, et al.
Pubblicazione: (2026)

Identifying Performance-Sensitive Configurations in Software Systems through Code Analysis with LLM Agents
di: Wang, Zehao, et al.
Pubblicazione: (2024)

From Helpful to Trustworthy: LLM Agents for Pair Programming
di: Ayon, Ragib Shahariar
Pubblicazione: (2026)

ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox
di: Li, Yuanyang, et al.
Pubblicazione: (2026)