Evaluating LLaMA 3.2 for Software Vulnerability Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909533549363200 |
|---|---|
| author | Gonçalves, José Silva, Miguel Cabral, Bernardo Dias, Tiago Maia, Eva Praça, Isabel Severino, Ricardo Ferreira, Luís Lino |
| author_facet | Gonçalves, José Silva, Miguel Cabral, Bernardo Dias, Tiago Maia, Eva Praça, Isabel Severino, Ricardo Ferreira, Luís Lino |
| contents | Deep Learning (DL) has emerged as a powerful tool for vulnerability detection, often outperforming traditional solutions. However, developing effective DL models requires large amounts of real-world data, which can be difficult to obtain in sufficient quantities. To address this challenge, DiverseVul dataset has been curated as the largest dataset of vulnerable and non-vulnerable C/C++ functions extracted exclusively from real-world projects. Its goal is to provide high-quality, large-scale samples for training DL models. However, during our study several inconsistencies were identified in the raw dataset while applying pre-processing techniques, highlighting the need for a refined version. In this work, we present a refined version of DiverseVul dataset, which is used to fine-tune a large language model, LLaMA 3.2, for vulnerability detection. Experimental results show that the use of pre-processing techniques led to an improvement in performance, with the model achieving an F1-Score of 66%, a competitive result when compared to our baseline, which achieved a 47% F1-Score in software vulnerability detection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07770 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating LLaMA 3.2 for Software Vulnerability Detection Gonçalves, José Silva, Miguel Cabral, Bernardo Dias, Tiago Maia, Eva Praça, Isabel Severino, Ricardo Ferreira, Luís Lino Machine Learning Artificial Intelligence Cryptography and Security Software Engineering Deep Learning (DL) has emerged as a powerful tool for vulnerability detection, often outperforming traditional solutions. However, developing effective DL models requires large amounts of real-world data, which can be difficult to obtain in sufficient quantities. To address this challenge, DiverseVul dataset has been curated as the largest dataset of vulnerable and non-vulnerable C/C++ functions extracted exclusively from real-world projects. Its goal is to provide high-quality, large-scale samples for training DL models. However, during our study several inconsistencies were identified in the raw dataset while applying pre-processing techniques, highlighting the need for a refined version. In this work, we present a refined version of DiverseVul dataset, which is used to fine-tune a large language model, LLaMA 3.2, for vulnerability detection. Experimental results show that the use of pre-processing techniques led to an improvement in performance, with the model achieving an F1-Score of 66%, a competitive result when compared to our baseline, which achieved a 47% F1-Score in software vulnerability detection. |
| title | Evaluating LLaMA 3.2 for Software Vulnerability Detection |
| topic | Machine Learning Artificial Intelligence Cryptography and Security Software Engineering |
| url | https://arxiv.org/abs/2503.07770 |