Security Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915265939243008 |
|---|---|
| author | Soltaniani, Farnaz Ghafari, Mohammad Sayagh, Mohammed |
| author_facet | Soltaniani, Farnaz Ghafari, Mohammad Sayagh, Mohammed |
| contents | Early detection of security bug reports (SBRs) is crucial for preventing vulnerabilities and ensuring system reliability. While machine learning models have been developed for SBR prediction, their predictive performance still has room for improvement. In this study, we conduct a comprehensive comparison between BERT and Random Forest (RF), a competitive baseline for predicting SBRs. The results show that RF outperforms BERT with a 34% higher average G-measure for within-project predictions. Adding only SBRs from various projects improves both models' average performance. However, including both security and nonsecurity bug reports significantly reduces RF's average performance to 46%, while boosts BERT to its best average performance of 66%, surpassing RF. In cross-project SBR prediction, BERT achieves a remarkable 62% G-measure, which is substantially higher than RF. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_21037 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Security Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest Soltaniani, Farnaz Ghafari, Mohammad Sayagh, Mohammed Cryptography and Security Artificial Intelligence Early detection of security bug reports (SBRs) is crucial for preventing vulnerabilities and ensuring system reliability. While machine learning models have been developed for SBR prediction, their predictive performance still has room for improvement. In this study, we conduct a comprehensive comparison between BERT and Random Forest (RF), a competitive baseline for predicting SBRs. The results show that RF outperforms BERT with a 34% higher average G-measure for within-project predictions. Adding only SBRs from various projects improves both models' average performance. However, including both security and nonsecurity bug reports significantly reduces RF's average performance to 46%, while boosts BERT to its best average performance of 66%, surpassing RF. In cross-project SBR prediction, BERT achieves a remarkable 62% G-measure, which is substantially higher than RF. |
| title | Security Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2504.21037 |