Cover Image

SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xia, Hongfei, Wang, Hongru, Liu, Zeming, Yu, Qian, Guo, Yuhang, Wang, Haifeng
Format:	Preprint
Published:	2025
Subjects:	Cryptography and Security Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Clawdrain: Exploiting Tool-Calling Chains for Stealthy Token Exhaustion in OpenClaw Agents
by: Dong, Ben, et al.
Published: (2026)

Evaluating Tool Cloning in Agentic-AI Ecosystems
by: Kim, Taein, et al.
Published: (2026)

AutoDFBench 1.0: A Benchmarking Framework for Digital Forensic Tool Testing and Generated Code Evaluation
by: Wickramasekara, Akila, et al.
Published: (2025)

Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safety
by: Chen, Xuan, et al.
Published: (2026)

PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilities
by: Wei, Zichao, et al.
Published: (2025)

Unity is Strength: Enhancing Precision in Reentrancy Vulnerability Detection of Smart Contract Analysis Tools
by: Wang, Zexu, et al.
Published: (2024)

From Obfuscated to Obvious: A Comprehensive JavaScript Deobfuscation Tool for Security Analysis
by: Zhou, Dongchao, et al.
Published: (2025)

Auditing MCP Servers for Over-Privileged Tool Capabilities
by: Huang, Charoes, et al.
Published: (2026)

Are AI-assisted Development Tools Immune to Prompt Injection?
by: Huang, Charoes, et al.
Published: (2026)

SBOM Generation Tools in the Python Ecosystem: an In-Detail Analysis
by: Cofano, Serena, et al.
Published: (2024)

PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
by: Deng, Gelei, et al.
Published: (2023)

Guardrails as Infrastructure: Policy-First Control for Tool-Orchestrated Workflows
by: Sigdel, Akshey, et al.
Published: (2026)

"I Don't Use AI for Everything": Exploring Utility, Attitude, and Responsibility of AI-empowered Tools in Software Development
by: Pan, Shidong, et al.
Published: (2024)

SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
by: Hu, Qi, et al.
Published: (2026)

Centralized Defense: Logging and Mitigation of Kubernetes Misconfigurations with Open Source Tools
by: Russell, Eoghan, et al.
Published: (2024)

Building a Cybersecurity Risk Metamodel for Improved Method and Tool Integration
by: Ponsard, Christophe
Published: (2024)

A Broad Comparative Evaluation of Software Debloating Tools
by: Brown, Michael D., et al.
Published: (2023)

HyperPUT: Generating Synthetic Faulty Programs to Challenge Bug-Finding Tools
by: Felici, Riccardo, et al.
Published: (2022)

Smart Contract and DeFi Security Tools: Do They Meet the Needs of Practitioners?
by: Chaliasos, Stefanos, et al.
Published: (2023)

RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
by: Wang, Yanlin, et al.
Published: (2026)

UniBOM -- A Unified SBOM Analysis and Visualisation Tool for IoT Systems and Beyond
by: Safronov, Vadim, et al.
Published: (2025)

MCP-SandboxScan: WASM-based Secure Execution and Runtime Analysis for MCP Tools
by: Tan, Zhuoran, et al.
Published: (2026)

Model Context Protocol Threat Modeling and Analyzing Vulnerabilities to Prompt Injection with Tool Poisoning
by: Huang, Charoes, et al.
Published: (2026)

HarnessAgent: Scaling Automatic Fuzzing Harness Construction with Tool-Augmented LLM Pipelines
by: Yang, Kang, et al.
Published: (2025)

ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents
by: Wu, Jiangrong, et al.
Published: (2026)

IssueGuard: Real-Time Secret Leak Prevention Tool for GitHub Issue Reports
by: Rahman, Md Nafiu, et al.
Published: (2026)

Many Tools, Few Exploitable Vulnerabilities: A Survey of 246 Static Code Analyzers for Security
by: Hermann, Kevin, et al.
Published: (2026)

RiskHarvester: A Risk-based Tool to Prioritize Secret Removal Efforts in Software Artifacts
by: Basak, Setu Kumar, et al.
Published: (2025)

AssetHarvester: A Static Analysis Tool for Detecting Secret-Asset Pairs in Software Artifacts
by: Basak, Setu Kumar, et al.
Published: (2024)

NESSiE: The Necessary Safety Benchmark -- Identifying Errors that should not Exist
by: Bertram, Johannes, et al.
Published: (2026)

From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection
by: Han, Junxiao, et al.
Published: (2025)

Evaluating Nova 2.0 Lite model under Amazon's Frontier Model Safety Framework
by: Krishna, Satyapriya, et al.
Published: (2026)

CNT: Safety-oriented Function Reuse across LLMs via Cross-Model Neuron Transfer
by: Zhao, Yue, et al.
Published: (2026)

VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specifications
by: Zhu, Hao, et al.
Published: (2025)

Jailbreak Distillation: Renewable Safety Benchmarking
by: Zhang, Jingyu, et al.
Published: (2025)

Decoding Secret Memorization in Code LLMs Through Token-Level Characterization
by: Nie, Yuqing, et al.
Published: (2024)

PrediQL: Automated Testing of GraphQL APIs with LLMs
by: Liu, Shaolun, et al.
Published: (2025)

When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
by: Wang, Su, et al.
Published: (2026)

Comparing Effectiveness and Efficiency of Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP) Tools in a Large Java-based System
by: Seth, Aishwarya, et al.
Published: (2023)

Guiding Symbolic Execution with Static Analysis and LLMs for Vulnerability Discovery
by: Shafiuzzaman, Md, et al.
Published: (2026)