Cover Image

Benchmarks for Trajectory Safety Evaluation and Diagnosis in OpenClaw and Codex: ATBench-Claw and ATBench-Codex

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Zhonghao, Li, Yu, Zhu, Yanxu, Zhou, Tianyi, Xie, Yuejin, Luo, Haoyu, Shao, Jing, Hu, Xia, Liu, Dongrui
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
by: Li, Yu, et al.
Published: (2026)

Clawdrain: Exploiting Tool-Calling Chains for Stealthy Token Exhaustion in OpenClaw Agents
by: Dong, Ben, et al.
Published: (2026)

A Trajectory-Based Safety Audit of Clawdbot (OpenClaw)
by: Chen, Tianyu, et al.
Published: (2026)

Codexity: Secure AI-assisted Code Generation
by: Kim, Sung Yong, et al.
Published: (2024)

EvoClaw: Evaluating AI Agents on Continuous Software Evolution
by: Deng, Gangda, et al.
Published: (2026)

StatsClaw: An AI-Collaborative Workflow for Statistical Software Development
by: Qin, Tianzhu, et al.
Published: (2026)

Clawed and Dangerous: Can We Trust Open Agentic Systems?
by: Chen, Shiping, et al.
Published: (2026)

Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows
by: Li, Chenxin, et al.
Published: (2026)

Engineering Pitfalls in AI Coding Tools: An Empirical Study of Bugs in Claude Code, Codex, and Gemini CLI
by: Zhang, Ruixin, et al.
Published: (2026)

ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents
by: Meng, Fanqing, et al.
Published: (2026)

QuantClaw: Precision Where It Matters for OpenClaw
by: Zhang, Manyi, et al.
Published: (2026)

CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases
by: Liu, Xiangyan, et al.
Published: (2024)

MatClaw: An Autonomous Code-First LLM Agent for End-to-End Materials Exploration
by: Zhang, Chenmu, et al.
Published: (2026)

QASecClaw: A Multi-Agent LLM Approach for False Positive Reduction in Static Application Security Testing
by: Ameen, Mohd Ruhul, et al.
Published: (2026)

Piloting Copilot, Codex, and StarCoder2: Hot Temperature, Cold Prompts, or Black Magic?
by: Döderlein, Jean-Baptiste, et al.
Published: (2022)

NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition
by: Deng, Le, et al.
Published: (2025)

ClawKeeper: Comprehensive Safety Protection for OpenClaw Agents Through Skills, Plugins, and Watchers
by: Liu, Songyang, et al.
Published: (2026)

ClawWorm: Self-Propagating Attacks Across LLM Agent Ecosystems
by: Zhang, Yihao, et al.
Published: (2026)

Security, Privacy, and Ethical Risks in OpenClaw
by: Jin, Yutong, et al.
Published: (2026)

ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
by: Xie, Bang, et al.
Published: (2026)

A Systematic Security Evaluation of OpenClaw and Its Variants
by: Wang, Yuhang, et al.
Published: (2026)

CommitSuite: A Comprehensive Benchmark for Commit Classification and Message Generation
by: Wan, Zirui, et al.
Published: (2026)

Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
by: Harsh, Reetu Raj, et al.
Published: (2026)

Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw
by: Wang, Zijun, et al.
Published: (2026)

A Benchmark for Evaluating Repository-Level Code Agents with Intermediate Reasoning on Feature Addition Task
by: Liu, Shuhan, et al.
Published: (2026)

ClawVM: Harness-Managed Virtual Memory for Stateful Tool-Using LLM Agents
by: Rafique, Mofasshara, et al.
Published: (2026)

SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs
by: Xia, Hongfei, et al.
Published: (2025)

Assessing and Advancing Benchmarks for Evaluating Large Language Models in Software Engineering Tasks
by: Hu, Xing, et al.
Published: (2025)

DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents
by: Hong, Sirui, et al.
Published: (2026)

Re-Evaluating Code LLM Benchmarks Under Semantic Mutation
by: Pan, Zhiyuan, et al.
Published: (2025)

Voices from the Frontier: A Comprehensive Analysis of the OpenAI Developer Forum
by: Hou, Xinyi, et al.
Published: (2024)

ClawTrap: A MITM-Based Red-Teaming Framework for Real-World OpenClaw Security Evaluation
by: Zhao, Haochen, et al.
Published: (2026)

Open the Oyster: Empirical Evaluation and Improvement of Code Reasoning Confidence in LLMs
by: Wang, Shufan, et al.
Published: (2025)

Documenting Ethical Considerations in Open Source AI Models
by: Gao, Haoyu, et al.
Published: (2024)

SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
by: Ye, Zhifan, et al.
Published: (2025)

Psychological Safety Framework in Pull-based Open Source Projects
by: Sesari, Emeralda, et al.
Published: (2025)

On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source
by: Gao, Haoyu, et al.
Published: (2026)

Are Benchmark Tests Strong Enough? Mutation-Guided Diagnosis and Augmentation of Regression Suites
by: Li, Chenglin, et al.
Published: (2026)

Don't Let the Claw Grip Your Hand: A Security Analysis and Defense Framework for OpenClaw
by: Shan, Zhengyang, et al.
Published: (2026)

Geo-ATBench: A Benchmark for Geospatial Audio Tagging with Geospatial Semantic Context
by: Hou, Yuanbo, et al.
Published: (2026)