Risks of ignoring uncertainty propagation in AI-augmented security pipelines
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913945231556608 |
|---|---|
| author | Mezzi, Emanuele Papotti, Aurora Massacci, Fabio Tuma, Katja |
| author_facet | Mezzi, Emanuele Papotti, Aurora Massacci, Fabio Tuma, Katja |
| contents | The use of AI technologies is being integrated into the secure development of software-based systems, with an increasing trend of composing AI-based subsystems (with uncertain levels of performance) into automated pipelines. This presents a fundamental research challenge and seriously threatens safety-critical domains. Despite the existing knowledge about uncertainty in risk analysis, no previous work has estimated the uncertainty of AI-augmented systems given the propagation of errors in the pipeline. We provide the formal underpinnings for capturing uncertainty propagation, develop a simulator to quantify uncertainty, and evaluate the simulation of propagating errors with one case study. We discuss the generalizability of our approach and its limitations and present recommendations for evaluation policies concerning AI systems. Future work includes extending the approach by relaxing the remaining assumptions and by experimenting with a real system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14540 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Risks of ignoring uncertainty propagation in AI-augmented security pipelines Mezzi, Emanuele Papotti, Aurora Massacci, Fabio Tuma, Katja Software Engineering Artificial Intelligence Cryptography and Security The use of AI technologies is being integrated into the secure development of software-based systems, with an increasing trend of composing AI-based subsystems (with uncertain levels of performance) into automated pipelines. This presents a fundamental research challenge and seriously threatens safety-critical domains. Despite the existing knowledge about uncertainty in risk analysis, no previous work has estimated the uncertainty of AI-augmented systems given the propagation of errors in the pipeline. We provide the formal underpinnings for capturing uncertainty propagation, develop a simulator to quantify uncertainty, and evaluate the simulation of propagating errors with one case study. We discuss the generalizability of our approach and its limitations and present recommendations for evaluation policies concerning AI systems. Future work includes extending the approach by relaxing the remaining assumptions and by experimenting with a real system. |
| title | Risks of ignoring uncertainty propagation in AI-augmented security pipelines |
| topic | Software Engineering Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2407.14540 |