Confronting the Reproducibility Crisis: A Case Study of Challenges in Cybersecurity AI
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915063143596032 |
|---|---|
| author | Moulton, Richard H. McCully, Gary A. Hastings, John D. |
| author_facet | Moulton, Richard H. McCully, Gary A. Hastings, John D. |
| contents | In the rapidly evolving field of cybersecurity, ensuring the reproducibility of AI-driven research is critical to maintaining the reliability and integrity of security systems. This paper addresses the reproducibility crisis within the domain of adversarial robustness -- a key area in AI-based cybersecurity that focuses on defending deep neural networks against malicious perturbations. Through a detailed case study, we attempt to validate results from prior work on certified robustness using the VeriGauge toolkit, revealing significant challenges due to software and hardware incompatibilities, version conflicts, and obsolescence. Our findings underscore the urgent need for standardized methodologies, containerization, and comprehensive documentation to ensure the reproducibility of AI models deployed in critical cybersecurity applications. By tackling these reproducibility challenges, we aim to contribute to the broader discourse on securing AI systems against advanced persistent threats, enhancing network and IoT security, and protecting critical infrastructure. This work advocates for a concerted effort within the research community to prioritize reproducibility, thereby strengthening the foundation upon which future cybersecurity advancements are built. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_18753 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Confronting the Reproducibility Crisis: A Case Study of Challenges in Cybersecurity AI Moulton, Richard H. McCully, Gary A. Hastings, John D. Machine Learning Artificial Intelligence Cryptography and Security I.2.0; D.4.6; K.6.5; I.5.1 In the rapidly evolving field of cybersecurity, ensuring the reproducibility of AI-driven research is critical to maintaining the reliability and integrity of security systems. This paper addresses the reproducibility crisis within the domain of adversarial robustness -- a key area in AI-based cybersecurity that focuses on defending deep neural networks against malicious perturbations. Through a detailed case study, we attempt to validate results from prior work on certified robustness using the VeriGauge toolkit, revealing significant challenges due to software and hardware incompatibilities, version conflicts, and obsolescence. Our findings underscore the urgent need for standardized methodologies, containerization, and comprehensive documentation to ensure the reproducibility of AI models deployed in critical cybersecurity applications. By tackling these reproducibility challenges, we aim to contribute to the broader discourse on securing AI systems against advanced persistent threats, enhancing network and IoT security, and protecting critical infrastructure. This work advocates for a concerted effort within the research community to prioritize reproducibility, thereby strengthening the foundation upon which future cybersecurity advancements are built. |
| title | Confronting the Reproducibility Crisis: A Case Study of Challenges in Cybersecurity AI |
| topic | Machine Learning Artificial Intelligence Cryptography and Security I.2.0; D.4.6; K.6.5; I.5.1 |
| url | https://arxiv.org/abs/2405.18753 |