On the Proactive Generation of Unsafe Images From Text-To-Image Models Using Benign Prompts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917912792530944 |
|---|---|
| author | Wu, Yixin Yu, Ning Backes, Michael Shen, Yun Zhang, Yang |
| author_facet | Wu, Yixin Yu, Ning Backes, Michael Shen, Yun Zhang, Yang |
| contents | Malicious or manipulated prompts are known to exploit text-to-image models to generate unsafe images. Existing studies, however, focus on the passive exploitation of such harmful capabilities. In this paper, we investigate the proactive generation of unsafe images from benign prompts (e.g., a photo of a cat) through maliciously modified text-to-image models. Our preliminary investigation demonstrates that poisoning attacks are a viable method to achieve this goal but uncovers significant side effects, where unintended spread to non-targeted prompts compromises attack stealthiness. Root cause analysis identifies conceptual similarity as an important contributing factor to these side effects. To address this, we propose a stealthy poisoning attack method that balances covertness and performance. Our findings highlight the potential risks of adopting text-to-image models in real-world scenarios, thereby calling for future research and safety measures in this space. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_16613 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On the Proactive Generation of Unsafe Images From Text-To-Image Models Using Benign Prompts Wu, Yixin Yu, Ning Backes, Michael Shen, Yun Zhang, Yang Cryptography and Security Malicious or manipulated prompts are known to exploit text-to-image models to generate unsafe images. Existing studies, however, focus on the passive exploitation of such harmful capabilities. In this paper, we investigate the proactive generation of unsafe images from benign prompts (e.g., a photo of a cat) through maliciously modified text-to-image models. Our preliminary investigation demonstrates that poisoning attacks are a viable method to achieve this goal but uncovers significant side effects, where unintended spread to non-targeted prompts compromises attack stealthiness. Root cause analysis identifies conceptual similarity as an important contributing factor to these side effects. To address this, we propose a stealthy poisoning attack method that balances covertness and performance. Our findings highlight the potential risks of adopting text-to-image models in real-world scenarios, thereby calling for future research and safety measures in this space. |
| title | On the Proactive Generation of Unsafe Images From Text-To-Image Models Using Benign Prompts |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2310.16613 |