Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909976459476992 |
|---|---|
| author | Verma, Apurv Krishna, Satyapriya Gehrmann, Sebastian Seshadri, Madhavan Pradhan, Anu Ault, Tom Barrett, Leslie Rabinowitz, David Doucette, John Phan, NhatHai |
| author_facet | Verma, Apurv Krishna, Satyapriya Gehrmann, Sebastian Seshadri, Madhavan Pradhan, Anu Ault, Tom Barrett, Leslie Rabinowitz, David Doucette, John Phan, NhatHai |
| contents | Creating secure and resilient applications with large language models (LLM) requires anticipating, adjusting to, and countering unforeseen threats. Red-teaming has emerged as a critical technique for identifying vulnerabilities in real-world LLM implementations. This paper presents a detailed threat model and provides a systematization of knowledge (SoK) of red-teaming attacks on LLMs. We develop a taxonomy of attacks based on the stages of the LLM development and deployment process and extract various insights from previous research. In addition, we compile methods for defense and practical red-teaming strategies for practitioners. By delineating prominent attack motifs and shedding light on various entry points, this paper provides a framework for improving the security and robustness of LLM-based systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14937 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs) Verma, Apurv Krishna, Satyapriya Gehrmann, Sebastian Seshadri, Madhavan Pradhan, Anu Ault, Tom Barrett, Leslie Rabinowitz, David Doucette, John Phan, NhatHai Computation and Language Cryptography and Security I.2.7 Creating secure and resilient applications with large language models (LLM) requires anticipating, adjusting to, and countering unforeseen threats. Red-teaming has emerged as a critical technique for identifying vulnerabilities in real-world LLM implementations. This paper presents a detailed threat model and provides a systematization of knowledge (SoK) of red-teaming attacks on LLMs. We develop a taxonomy of attacks based on the stages of the LLM development and deployment process and extract various insights from previous research. In addition, we compile methods for defense and practical red-teaming strategies for practitioners. By delineating prominent attack motifs and shedding light on various entry points, this paper provides a framework for improving the security and robustness of LLM-based systems. |
| title | Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs) |
| topic | Computation and Language Cryptography and Security I.2.7 |
| url | https://arxiv.org/abs/2407.14937 |