Copertina

Measuring Security Without Fooling Ourselves: Why Benchmarking Agents Is Hard

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Abdelnabi, Sahar, Hicks, Chris, Rieck, Konrad, Sadeghi, Ahmad-Reza
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Cryptography and Security Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs
di: Salem, Ahmed, et al.
Pubblicazione: (2026)

CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents
di: Emerson, Harry, et al.
Pubblicazione: (2024)

HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
di: Chen, Qirui, et al.
Pubblicazione: (2026)

No More, No Less: Task Alignment in Terminal Agents
di: Mavali, Sina, et al.
Pubblicazione: (2026)

Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studies
di: Nakamura, Mason, et al.
Pubblicazione: (2025)

SkillTester: Benchmarking Utility and Security of Agent Skills
di: Wang, Leye, et al.
Pubblicazione: (2026)

Measuring Safety Alignment Effects in Autonomous Security Agents
di: David, Isaac, et al.
Pubblicazione: (2026)

ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversations
di: Gomaa, Amr, et al.
Pubblicazione: (2025)

Towards Unifying Quantitative Security Benchmarking for Multi Agent Systems
di: Sharma, Gauri, et al.
Pubblicazione: (2025)

Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
di: Zhang, Hanrong, et al.
Pubblicazione: (2024)

Why LLMs Fail: A Failure Analysis and Partial Success Measurement for Automated Security Patch Generation
di: Al-Maamari, Amir
Pubblicazione: (2026)

WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
di: Evtimov, Ivan, et al.
Pubblicazione: (2025)

Secure On-Device Video OOD Detection Without Backpropagation
di: Li, Shawn, et al.
Pubblicazione: (2025)

ZORRO: Zero-Knowledge Robustness and Privacy for Split Learning (Full Version)
di: Sheybani, Nojan, et al.
Pubblicazione: (2025)

Fooling SHAP with Output Shuffling Attacks
di: Yuan, Jun, et al.
Pubblicazione: (2024)

SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
di: Shen, Chihao, et al.
Pubblicazione: (2025)

MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents
di: Zhang, Dongsen, et al.
Pubblicazione: (2025)

RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments
di: Fu, Yuchuan, et al.
Pubblicazione: (2025)

Too Easily Fooled? Prompt Injection Breaks LLMs on Frustratingly Simple Multiple-Choice Questions
di: Guo, Xuyang, et al.
Pubblicazione: (2025)

AI Agents May Always Fall for Prompt Injections
di: Abdelnabi, Sahar, et al.
Pubblicazione: (2026)

Security of AI Agents
di: He, Yifeng, et al.
Pubblicazione: (2024)

Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
di: Schmotz, David, et al.
Pubblicazione: (2026)

Firewalls to Secure Dynamic LLM Agentic Networks
di: Abdelnabi, Sahar, et al.
Pubblicazione: (2025)

Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Space
di: Vyas, Sanyam, et al.
Pubblicazione: (2024)

Less is more? Rewards in RL for Cyber Defence
di: Bates, Elizabeth, et al.
Pubblicazione: (2025)

Provably Secure Agent Guardrail
di: Wu, Benlong, et al.
Pubblicazione: (2026)

BadScientist: Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers?
di: Jiang, Fengqing, et al.
Pubblicazione: (2025)

Offensive Security for AI Systems: Concepts, Practices, and Applications
di: Harguess, Josh, et al.
Pubblicazione: (2025)

Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark
di: Shao, Minghao, et al.
Pubblicazione: (2025)

Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis
di: Li, Zhiyuan, et al.
Pubblicazione: (2026)

Parallax: Why AI Agents That Think Must Never Act
di: Fokou, Joel
Pubblicazione: (2026)

Agent Security is a Systems Problem
di: Christodorescu, Mihai, et al.
Pubblicazione: (2026)

Security of Internet of Agents: Attacks and Countermeasures
di: Wang, Yuntao, et al.
Pubblicazione: (2025)

Symbolic Guardrails for Domain-Specific Agents: Stronger Safety and Security Guarantees Without Sacrificing Utility
di: Hong, Yining, et al.
Pubblicazione: (2026)

aCAPTCHA: Verifying That an Entity Is a Capable Agent via Asymmetric Hardness
di: Xu, Zuyao, et al.
Pubblicazione: (2026)

Large Language Models for Security Operations Centers: A Comprehensive Survey
di: Habibzadeh, Ali, et al.
Pubblicazione: (2025)

RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
di: Arzanipour, Atousa, et al.
Pubblicazione: (2025)

Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents
di: Puppala, Sai, et al.
Pubblicazione: (2026)

AgentWard: A Lifecycle Security Architecture for Autonomous AI Agents
di: Zhang, Yixiang, et al.
Pubblicazione: (2026)

Agent Audit: A Security Analysis System for LLM Agent Applications
di: Zhang, Haiyue, et al.
Pubblicazione: (2026)