Cover Image

Closing the Gap: A User Study on the Real-world Usefulness of AI-powered Vulnerability Detection & Repair in the IDE

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Steenhoek, Benjamin, Sivaraman, Kalpathy, Gonzalez, Renata Saldivar, Mohylevskyy, Yevhen, Moghaddam, Roshanak Zilouchian, Le, Wei
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Cryptography and Security Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents
by: Kozak, Matous, et al.
Published: (2025)

FaultLine: Automated Proof-of-Vulnerability Generation Using LLM Agents
by: Nitin, Vikram, et al.
Published: (2025)

PerfBench: Can Agents Resolve Real-World Performance Bugs?
by: Garg, Spandan, et al.
Published: (2025)

Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming
by: Agarwal, Anisha, et al.
Published: (2024)

RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot
by: Garg, Spandan, et al.
Published: (2023)

The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason
by: Liang, Shanchao, et al.
Published: (2025)

SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments
by: Arora, Avi, et al.
Published: (2025)

To Err is Machine: Vulnerability Detection Challenges LLM Reasoning
by: Steenhoek, Benjamin, et al.
Published: (2024)

AutoDev: Automated AI-Driven Development
by: Tufano, Michele, et al.
Published: (2024)

ContractTinker: LLM-Empowered Vulnerability Repair for Real-World Smart Contracts
by: Wang, Che, et al.
Published: (2024)

Root-Cause-Driven Automated Vulnerability Repair
by: Wang, Hulin, et al.
Published: (2026)

Towards Demystifying and Repairing LLM-in-the-Loop Vulnerabilities
by: Ma, Yujie, et al.
Published: (2026)

VulKey: Automated Vulnerability Repair Guided by Domain-Specific Repair Patterns
by: Li, Jia, et al.
Published: (2026)

Bridging the Gap: A Study of AI-based Vulnerability Management between Industry and Academia
by: Wan, Shengye, et al.
Published: (2024)

VulnRepairEval: An Exploit-Based Evaluation Framework for Assessing Large Language Model Vulnerability Repair Capabilities
by: Wang, Weizhe, et al.
Published: (2025)

Unlocking User-oriented Pages: Intention-driven Black-box Scanner for Real-world Web Applications
by: Wang, Weizhe, et al.
Published: (2025)

Semantics-Aligned, Curriculum-Driven, and Reasoning-Enhanced Vulnerability Repair Framework
by: Yang, Chengran, et al.
Published: (2025)

PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilities
by: Wei, Zichao, et al.
Published: (2025)

LLM4CVE: Enabling Iterative Automated Vulnerability Repair with Large Language Models
by: Fakih, Mohamad, et al.
Published: (2025)

RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code
by: Gautam, Dhruv, et al.
Published: (2025)

Fixing Security Vulnerabilities with AI in OSS-Fuzz
by: Zhang, Yuntong, et al.
Published: (2024)

Secure Coding with AI -- From Detection to Repair
by: Belozerov, Vladislav, et al.
Published: (2025)

Uncovering Hidden Inclusions of Vulnerable Dependencies in Real-World Java Projects
by: Schott, Stefan, et al.
Published: (2026)

Real-World Usability of Vulnerability Proof-of-Concepts: A Comprehensive Study
by: Dang, Wenjing, et al.
Published: (2025)

ACFIX: Guiding LLMs with Mined Common RBAC Practices for Context-Aware Repair of Access Control Vulnerabilities in Smart Contracts
by: Zhang, Lyuye, et al.
Published: (2024)

Code Security Vulnerability Repair Using Reinforcement Learning with Large Language Models
by: Islam, Nafis Tanveer, et al.
Published: (2024)

AI-Based Software Vulnerability Detection: A Systematic Literature Review
by: Shimmi, Samiha, et al.
Published: (2025)

Towards Understanding and Characterizing Vulnerabilities in Intelligent Connected Vehicles through Real-World Exploits
by: Wang, Yuelin, et al.
Published: (2026)

What Makes a Good LLM Agent for Real-world Penetration Testing?
by: Deng, Gelei, et al.
Published: (2026)

Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study
by: Le, Triet H. M., et al.
Published: (2024)

APPATCH: Automated Adaptive Prompting Large Language Models for Real-World Software Vulnerability Patching
by: Nong, Yu, et al.
Published: (2024)

SecureFixAgent: A Hybrid LLM Agent for Automated Python Static Vulnerability Repair
by: Gajjar, Jugal, et al.
Published: (2025)

On the Effectiveness of Function-Level Vulnerability Detectors for Inter-Procedural Vulnerabilities
by: Li, Zhen, et al.
Published: (2024)

MAS-SZZ: Multi-Agentic SZZ Algorithm for Vulnerability-Inducing Commit Identification
by: Cao, Sicong, et al.
Published: (2026)

KernJC: Automated Vulnerable Environment Generation for Linux Kernel Vulnerabilities
by: Ruan, Bonan, et al.
Published: (2024)

Evaluating LLMs for One-Shot Patching of Real and Artificial Vulnerabilities
by: Garg, Aayush, et al.
Published: (2025)

Mapping NVD Records to Their Vulnerability-fixing Commits: How Hard is It?
by: Nguyen, Huu Hung, et al.
Published: (2025)

Vulnerability-Hunter: An Adaptive Feature Perception Attention Network for Smart Contract Vulnerabilities
by: Chen, Yizhou
Published: (2024)

Game Rewards Vulnerabilities: Software Vulnerability Detection with Zero-Sum Game and Prototype Learning
by: Wen, Xin-Cheng, et al.
Published: (2024)

LuaTaint: A Static Analysis System for Web Configuration Interface Vulnerability of Internet of Things Devices
by: Xiang, Jiahui, et al.
Published: (2024)