Cover Image

Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shao, Minghao, Rani, Nanda, Milner, Kimberly, Xi, Haoran, Udeshi, Meet, Aggarwal, Saksham, Putrevu, Venkata Sai Charan, Shukla, Sandeep Kumar, Krishnamurthy, Prashanth, Khorrami, Farshad, Karri, Ramesh, Shafique, Muhammad
Format:	Preprint
Published:	2025
Subjects:	Cryptography and Security Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking
by: Rani, Nanda, et al.
Published: (2026)

D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System with Planner and Heterogeneous Executors for Offensive Security
by: Udeshi, Meet, et al.
Published: (2025)

CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution
by: Shao, Minghao, et al.
Published: (2025)

SaMOSA: Sandbox for Malware Orchestration and Side-Channel Analysis
by: Udeshi, Meet, et al.
Published: (2025)

AI In Cybersecurity Education -- Scalable Agentic CTF Design Principles and Educational Outcomes
by: Xi, Haoran, et al.
Published: (2026)

Safeguarding LLMs Against Misuse and AI-Driven Malware Using Steganographic Canaries
by: Raz, Md, et al.
Published: (2026)

NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security
by: Shao, Minghao, et al.
Published: (2024)

Binary Diff Summarization using Large Language Models
by: Udeshi, Meet, et al.
Published: (2025)

Ransomware 3.0: Self-Composing and LLM-Orchestrated
by: Raz, Md, et al.
Published: (2025)

REMaQE: Reverse Engineering Math Equations from Executables
by: Udeshi, Meet, et al.
Published: (2023)

SHIELD: A Host-Independent Framework for Ransomware Detection using Deep Filesystem Features
by: Raz, Md, et al.
Published: (2025)

Enabling Deep Visibility into VxWorks-Based Embedded Controllers in Cyber-Physical Systems for Anomaly Detection
by: Krishnamurthy, Prashanth, et al.
Published: (2025)

SCAMPER -- Synchrophasor Covert chAnnel for Malicious and Protective ERrands
by: Krishnamurthy, Prashanth, et al.
Published: (2025)

Real-Time Multi-Modal Subcomponent-Level Measurements for Trustworthy System Monitoring and Malware Detection
by: Khorrami, Farshad, et al.
Published: (2025)

Tracking Real-time Anomalies in Cyber-Physical Systems Through Dynamic Behavioral Analysis
by: Krishnamurthy, Prashanth, et al.
Published: (2024)

HiFi-CS: Towards Open Vocabulary Visual Grounding For Robotic Grasping Using Vision-Language Models
by: Bhat, Vineet, et al.
Published: (2024)

PrefixLLM: LLM-aided Prefix Circuit Design
by: Xiao, Weihua, et al.
Published: (2024)

RESCORE: LLM-Driven Simulation Recovery in Control Systems Research Papers
by: Bhat, Vineet, et al.
Published: (2026)

EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities
by: Abramovich, Talor, et al.
Published: (2024)

A Comprehensive Analysis of Machine Learning Based File Trap Selection Methods to Detect Crypto Ransomware
by: Putrevu, Mohan Anand, et al.
Published: (2024)

MapleGrasp: Mask-guided Feature Pooling for Language-driven Efficient Robotic Grasping
by: Bhat, Vineet, et al.
Published: (2025)

SENTAUR: Security EnhaNced Trojan Assessment Using LLMs Against Undesirable Revisions
by: Bhandari, Jitendra, et al.
Published: (2024)

Data-Efficient System Identification via Lipschitz Neural Networks
by: Wei, Shiqing, et al.
Published: (2024)

RAZER: Robust Accelerated Zero-Shot 3D Open-Vocabulary Panoptic Reconstruction with Spatio-Temporal Aggregation
by: Patel, Naman, et al.
Published: (2025)

Confidence-Aware Safe and Stable Control of Control-Affine Systems
by: Wei, Shiqing, et al.
Published: (2024)

Combining Switching Mechanism with Re-Initialization and Anomaly Detection for Resiliency of Cyber-Physical Systems
by: Fu, Hao, et al.
Published: (2024)

Prescribed-Time Stability Properties of Interconnected Systems
by: Krishnamurthy, Prashanth, et al.
Published: (2024)

Learning a Better Control Barrier Function Under Uncertain Dynamics
by: Dai, Bolun, et al.
Published: (2023)

Robust Neural Lyapunov Control for Nonlinear Systems With Quadratically Bounded Disturbances
by: Shiqing Wei, et al.
Published: (2025)

MultiTalk: Introspective and Extrospective Dialogue for Human-Environment-LLM Alignment
by: Devarakonda, Venkata Naren, et al.
Published: (2024)

Grounding LLMs For Robot Task Planning Using Closed-loop State Feedback
by: Bhat, Vineet, et al.
Published: (2024)

3D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Tasks
by: Bhat, Vineet, et al.
Published: (2025)

Grounding Large Language Models for Robot Task Planning Using Closed‐Loop State Feedback
by: Vineet Bhat, et al.
Published: (2025)

A Control Barrier Function-Constrained Model Predictive Control Framework for Safe Reinforcement Learning
by: Kaypak, Ali Umut, et al.
Published: (2026)

Sailing Through Point Clouds: Safe Navigation Using Point Cloud Based Control Barrier Functions
by: Dai, Bolun, et al.
Published: (2024)

Out-of-Distribution Detection with Overlap Index
by: Fu, Hao, et al.
Published: (2024)

An Upper Bound for the Distribution Overlap Index and Its Applications
by: Fu, Hao, et al.
Published: (2022)

Differentiable Optimization Based Time-Varying Control Barrier Functions for Dynamic Obstacle Avoidance
by: Dai, Bolun, et al.
Published: (2023)

CLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoring
by: Fu, Hao, et al.
Published: (2024)

Proactive Hierarchical Control Barrier Function-Based Safety Prioritization in Close Human-Robot Interaction Scenarios
by: Maithani, Patanjali, et al.
Published: (2025)