Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915045673271296 |
|---|---|
| author | Wang, Tony T. Hughes, John Sleight, Henry Schaeffer, Rylan Agrawal, Rajashree Barez, Fazl Sharma, Mrinank Mu, Jesse Shavit, Nir Perez, Ethan |
| author_facet | Wang, Tony T. Hughes, John Sleight, Henry Schaeffer, Rylan Agrawal, Rajashree Barez, Fazl Sharma, Mrinank Mu, Jesse Shavit, Nir Perez, Ethan |
| contents | Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem. In this paper, we investigate the difficulty of jailbreak-defense when we only want to forbid a narrowly-defined set of behaviors. As a case study, we focus on preventing an LLM from helping a user make a bomb. We find that popular defenses such as safety training, adversarial training, and input/output classifiers are unable to fully solve this problem. In pursuit of a better solution, we develop a transcript-classifier defense which outperforms the baseline defenses we test. However, our classifier defense still fails in some circumstances, which highlights the difficulty of jailbreak-defense even in a narrow domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_02159 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach Wang, Tony T. Hughes, John Sleight, Henry Schaeffer, Rylan Agrawal, Rajashree Barez, Fazl Sharma, Mrinank Mu, Jesse Shavit, Nir Perez, Ethan Machine Learning Artificial Intelligence Computation and Language Cryptography and Security Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem. In this paper, we investigate the difficulty of jailbreak-defense when we only want to forbid a narrowly-defined set of behaviors. As a case study, we focus on preventing an LLM from helping a user make a bomb. We find that popular defenses such as safety training, adversarial training, and input/output classifiers are unable to fully solve this problem. In pursuit of a better solution, we develop a transcript-classifier defense which outperforms the baseline defenses we test. However, our classifier defense still fails in some circumstances, which highlights the difficulty of jailbreak-defense even in a narrow domain. |
| title | Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach |
| topic | Machine Learning Artificial Intelligence Computation and Language Cryptography and Security |
| url | https://arxiv.org/abs/2412.02159 |