JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913569233174528 |
|---|---|
| author | Chao, Patrick Debenedetti, Edoardo Robey, Alexander Andriushchenko, Maksym Croce, Francesco Sehwag, Vikash Dobriban, Edgar Flammarion, Nicolas Pappas, George J. Tramer, Florian Hassani, Hamed Wong, Eric |
| author_facet | Chao, Patrick Debenedetti, Edoardo Robey, Alexander Andriushchenko, Maksym Croce, Francesco Sehwag, Vikash Dobriban, Edgar Flammarion, Nicolas Pappas, George J. Tramer, Florian Hassani, Hamed Wong, Eric |
| contents | Jailbreak attacks cause large language models (LLMs) to generate harmful, unethical, or otherwise objectionable content. Evaluating these attacks presents a number of challenges, which the current collection of benchmarks and evaluation techniques do not adequately address. First, there is no clear standard of practice regarding jailbreaking evaluation. Second, existing works compute costs and success rates in incomparable ways. And third, numerous works are not reproducible, as they withhold adversarial prompts, involve closed-source code, or rely on evolving proprietary APIs. To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (2) a jailbreaking dataset comprising 100 behaviors -- both original and sourced from prior work (Zou et al., 2023; Mazeika et al., 2023, 2024) -- which align with OpenAI's usage policies; (3) a standardized evaluation framework at https://github.com/JailbreakBench/jailbreakbench that includes a clearly defined threat model, system prompts, chat templates, and scoring functions; and (4) a leaderboard at https://jailbreakbench.github.io/ that tracks the performance of attacks and defenses for various LLMs. We have carefully considered the potential ethical implications of releasing this benchmark, and believe that it will be a net positive for the community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models Chao, Patrick Debenedetti, Edoardo Robey, Alexander Andriushchenko, Maksym Croce, Francesco Sehwag, Vikash Dobriban, Edgar Flammarion, Nicolas Pappas, George J. Tramer, Florian Hassani, Hamed Wong, Eric Cryptography and Security Machine Learning Jailbreak attacks cause large language models (LLMs) to generate harmful, unethical, or otherwise objectionable content. Evaluating these attacks presents a number of challenges, which the current collection of benchmarks and evaluation techniques do not adequately address. First, there is no clear standard of practice regarding jailbreaking evaluation. Second, existing works compute costs and success rates in incomparable ways. And third, numerous works are not reproducible, as they withhold adversarial prompts, involve closed-source code, or rely on evolving proprietary APIs. To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (2) a jailbreaking dataset comprising 100 behaviors -- both original and sourced from prior work (Zou et al., 2023; Mazeika et al., 2023, 2024) -- which align with OpenAI's usage policies; (3) a standardized evaluation framework at https://github.com/JailbreakBench/jailbreakbench that includes a clearly defined threat model, system prompts, chat templates, and scoring functions; and (4) a leaderboard at https://jailbreakbench.github.io/ that tracks the performance of attacks and defenses for various LLMs. We have carefully considered the potential ethical implications of releasing this benchmark, and believe that it will be a net positive for the community. |
| title | JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2404.01318 |