Sparad:
| Huvudupphovsmän: | , , |
|---|---|
| Materialtyp: | Recurso digital |
| Språk: | engelska |
| Publicerad: |
Zenodo
2025
|
| Ämnen: | |
| Länkar: | https://doi.org/10.5281/zenodo.17868180 |
| Taggar: |
Lägg till en tagg
Inga taggar, Lägg till första taggen!
|
Innehållsförteckning:
- <h1>AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package</h1> <p>This repository accompanies the paper “Experimental Evaluation of AI-Augmented Cybersecurity Requirements Generation Leveraging LLMs’ Capabilities” (<a href="https://doi.org/10.1109/ACCESS.2026.3658339">10.1109/ACCESS.2026.3658339</a>). It contains every script, dataset, prompt template and result needed to fully reproduce our empirical study.</p> <h2>Research Description</h2> <div> <div>This project investigates the practical use of state‑of‑the‑art Large Language Models (LLMs) to transform high‑level, standard‑driven cyber‑security controls into concrete, system‑specific requirements. Using a synthetic yet industrially plausible case study—AI4I4, an IoT‑enabled automotive logistics platform—we benchmark thirteen frontier models (GPT‑4, LLaMa 3, Mistral, QWen, etc.), representing tge state of the art as of September 2024, across four prompting pipelines and three temperature regimes.</div> <br> <div>Key contributions include:</div> <br> <div>1. <strong>Annotated benchmark</strong> of 54 ISO‑27002 clauses with placeholder semantics suitable for automatic instantiation.</div> <div>2. <strong>LangChain pipelines</strong> that decompose the task into applicability filtering, domain‑element search, requirement generation, and JSON formatting.</div> <div>3. <strong>Comprehensive evaluation</strong> of accuracy (precision, recall, F2), creativity (F2‑synthetic), and consistency (Jaccard overlap across runs).</div> <div>4. <strong>Prompt library</strong> enumerating >180 templates, showing how subtle changes in instruction design affect hallucination rate and coverage.</div> <br> <div>The artefacts and scripts below allow full replication—from raw prompts to final figures—on any infrastructure with access to the referenced models.</div> <div> </div> <div>For more information on the repository structure, reproducibility, licensing, and contact details, please refer to the README.</div> </div>