FLIRT: Feedback Loop In-context Red Teaming
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917830681690112 |
|---|---|
| author | Mehrabi, Ninareh Goyal, Palash Dupuy, Christophe Hu, Qian Ghosh, Shalini Zemel, Richard Chang, Kai-Wei Galstyan, Aram Gupta, Rahul |
| author_facet | Mehrabi, Ninareh Goyal, Palash Dupuy, Christophe Hu, Qian Ghosh, Shalini Zemel, Richard Chang, Kai-Wei Galstyan, Aram Gupta, Rahul |
| contents | Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become a priority. In this work, we propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation. In particular, taking text-to-image models as target models, we explore different feedback mechanisms to automatically learn effective and diverse adversarial prompts. Our experiments demonstrate that even with enhanced safety features, Stable Diffusion (SD) models are vulnerable to our adversarial prompts, raising concerns on their robustness in practical uses. Furthermore, we demonstrate that the proposed framework is effective for red teaming text-to-text models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_04265 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | FLIRT: Feedback Loop In-context Red Teaming Mehrabi, Ninareh Goyal, Palash Dupuy, Christophe Hu, Qian Ghosh, Shalini Zemel, Richard Chang, Kai-Wei Galstyan, Aram Gupta, Rahul Artificial Intelligence Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become a priority. In this work, we propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation. In particular, taking text-to-image models as target models, we explore different feedback mechanisms to automatically learn effective and diverse adversarial prompts. Our experiments demonstrate that even with enhanced safety features, Stable Diffusion (SD) models are vulnerable to our adversarial prompts, raising concerns on their robustness in practical uses. Furthermore, we demonstrate that the proposed framework is effective for red teaming text-to-text models. |
| title | FLIRT: Feedback Loop In-context Red Teaming |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2308.04265 |