Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agents
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912747274371072 |
|---|---|
| author | Widyasari, Ratnadira Weyssow, Martin Irsan, Ivana Clairine Ang, Han Wei Liauw, Frank Ouh, Eng Lieh Shar, Lwin Khin Kang, Hong Jin Lo, David |
| author_facet | Widyasari, Ratnadira Weyssow, Martin Irsan, Ivana Clairine Ang, Han Wei Liauw, Frank Ouh, Eng Lieh Shar, Lwin Khin Kang, Hong Jin Lo, David |
| contents | Detecting vulnerabilities in source code remains a critical yet challenging task, especially when benign and vulnerable functions share significant similarities. In this work, we introduce VulTrial, a courtroom-inspired multi-agent framework designed to identify vulnerable code and to provide explanations. It employs four role-specific agents, which are security researcher, code author, moderator, and review board. Using GPT-4o as the base LLM, VulTrial almost doubles the efficacy of prior best-performing baselines. Additionally, we show that role-specific instruction tuning with small quantities of data significantly further boosts VulTrial's efficacy. Our extensive experiments demonstrate the efficacy of VulTrial across different LLMs, including an open-source, in-house-deployable model (LLaMA-3.1-8B), as well as the high quality of its generated explanations and its ability to uncover multiple confirmed zero-day vulnerabilities in the wild. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10961 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agents Widyasari, Ratnadira Weyssow, Martin Irsan, Ivana Clairine Ang, Han Wei Liauw, Frank Ouh, Eng Lieh Shar, Lwin Khin Kang, Hong Jin Lo, David Software Engineering Artificial Intelligence Detecting vulnerabilities in source code remains a critical yet challenging task, especially when benign and vulnerable functions share significant similarities. In this work, we introduce VulTrial, a courtroom-inspired multi-agent framework designed to identify vulnerable code and to provide explanations. It employs four role-specific agents, which are security researcher, code author, moderator, and review board. Using GPT-4o as the base LLM, VulTrial almost doubles the efficacy of prior best-performing baselines. Additionally, we show that role-specific instruction tuning with small quantities of data significantly further boosts VulTrial's efficacy. Our extensive experiments demonstrate the efficacy of VulTrial across different LLMs, including an open-source, in-house-deployable model (LLaMA-3.1-8B), as well as the high quality of its generated explanations and its ability to uncover multiple confirmed zero-day vulnerabilities in the wild. |
| title | Let the Trial Begin: A Mock-Court Approach to Vulnerability Detection using LLM-Based Agents |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2505.10961 |