AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package
Fuente:
Zenodo
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Recurso digital |
| Lenguaje: | inglés |
| Publicado: |
Zenodo
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866901167779348480 |
|---|---|
| author | Yelmo, Juan Carlos Martín, Yod-Samuel Perez-Acuna, Santiago |
| author_facet | Yelmo, Juan Carlos Martín, Yod-Samuel Perez-Acuna, Santiago |
| contents | <h1>AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package</h1> <p>This repository accompanies the paper “Experimental Evaluation of AI-Augmented Cybersecurity Requirements Generation Leveraging LLMs’ Capabilities” (<a href="https://doi.org/10.1109/ACCESS.2026.3658339">10.1109/ACCESS.2026.3658339</a>). It contains every script, dataset, prompt template and result needed to fully reproduce our empirical study.</p> <h2>Research Description</h2> <div> <div>This project investigates the practical use of state‑of‑the‑art Large Language Models (LLMs) to transform high‑level, standard‑driven cyber‑security controls into concrete, system‑specific requirements. Using a synthetic yet industrially plausible case study—AI4I4, an IoT‑enabled automotive logistics platform—we benchmark thirteen frontier models (GPT‑4, LLaMa 3, Mistral, QWen, etc.), representing tge state of the art as of September 2024, across four prompting pipelines and three temperature regimes.</div> <br> <div>Key contributions include:</div> <br> <div>1. <strong>Annotated benchmark</strong> of 54 ISO‑27002 clauses with placeholder semantics suitable for automatic instantiation.</div> <div>2. <strong>LangChain pipelines</strong> that decompose the task into applicability filtering, domain‑element search, requirement generation, and JSON formatting.</div> <div>3. <strong>Comprehensive evaluation</strong> of accuracy (precision, recall, F2), creativity (F2‑synthetic), and consistency (Jaccard overlap across runs).</div> <div>4. <strong>Prompt library</strong> enumerating >180 templates, showing how subtle changes in instruction design affect hallucination rate and coverage.</div> <br> <div>The artefacts and scripts below allow full replication—from raw prompts to final figures—on any infrastructure with access to the referenced models.</div> <div> </div> <div>For more information on the repository structure, reproducibility, licensing, and contact details, please refer to the README.</div> </div> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17868180 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package Yelmo, Juan Carlos Martín, Yod-Samuel Perez-Acuna, Santiago LLM Software Engineering Requirements Engineering Cybersecurity Artificial Intelligence <h1>AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package</h1> <p>This repository accompanies the paper “Experimental Evaluation of AI-Augmented Cybersecurity Requirements Generation Leveraging LLMs’ Capabilities” (<a href="https://doi.org/10.1109/ACCESS.2026.3658339">10.1109/ACCESS.2026.3658339</a>). It contains every script, dataset, prompt template and result needed to fully reproduce our empirical study.</p> <h2>Research Description</h2> <div> <div>This project investigates the practical use of state‑of‑the‑art Large Language Models (LLMs) to transform high‑level, standard‑driven cyber‑security controls into concrete, system‑specific requirements. Using a synthetic yet industrially plausible case study—AI4I4, an IoT‑enabled automotive logistics platform—we benchmark thirteen frontier models (GPT‑4, LLaMa 3, Mistral, QWen, etc.), representing tge state of the art as of September 2024, across four prompting pipelines and three temperature regimes.</div> <br> <div>Key contributions include:</div> <br> <div>1. <strong>Annotated benchmark</strong> of 54 ISO‑27002 clauses with placeholder semantics suitable for automatic instantiation.</div> <div>2. <strong>LangChain pipelines</strong> that decompose the task into applicability filtering, domain‑element search, requirement generation, and JSON formatting.</div> <div>3. <strong>Comprehensive evaluation</strong> of accuracy (precision, recall, F2), creativity (F2‑synthetic), and consistency (Jaccard overlap across runs).</div> <div>4. <strong>Prompt library</strong> enumerating >180 templates, showing how subtle changes in instruction design affect hallucination rate and coverage.</div> <br> <div>The artefacts and scripts below allow full replication—from raw prompts to final figures—on any infrastructure with access to the referenced models.</div> <div> </div> <div>For more information on the repository structure, reproducibility, licensing, and contact details, please refer to the README.</div> </div> |
| title | AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package |
| topic | LLM Software Engineering Requirements Engineering Cybersecurity Artificial Intelligence |
| url | https://doi.org/10.5281/zenodo.17868180 |