Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911815153221632 |
|---|---|
| author | Rao, Abhinav Vashistha, Sachin Naik, Atharva Aditya, Somak Choudhury, Monojit |
| author_facet | Rao, Abhinav Vashistha, Sachin Naik, Atharva Aditya, Somak Choudhury, Monojit |
| contents | Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies. Limited studies have been conducted to formalize and analyze these attacks and their mitigations. We bridge this gap by proposing a formalism and a taxonomy of known (and possible) jailbreaks. We survey existing jailbreak methods and their effectiveness on open-source and commercial LLMs (such as GPT-based models, OPT, BLOOM, and FLAN-T5-XXL). We further discuss the challenges of jailbreak detection in terms of their effectiveness against known attacks. For further analysis, we release a dataset of model outputs across 3700 jailbreak prompts over 4 tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_14965 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks Rao, Abhinav Vashistha, Sachin Naik, Atharva Aditya, Somak Choudhury, Monojit Computation and Language Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies. Limited studies have been conducted to formalize and analyze these attacks and their mitigations. We bridge this gap by proposing a formalism and a taxonomy of known (and possible) jailbreaks. We survey existing jailbreak methods and their effectiveness on open-source and commercial LLMs (such as GPT-based models, OPT, BLOOM, and FLAN-T5-XXL). We further discuss the challenges of jailbreak detection in terms of their effectiveness against known attacks. For further analysis, we release a dataset of model outputs across 3700 jailbreak prompts over 4 tasks. |
| title | Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2305.14965 |