Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912519873888256 |
|---|---|
| author | Yang, Fan Huang, Yihao Zhu, Jiayi Shi, Ling Pu, Geguang Dong, Jin Song Wang, Kailong |
| author_facet | Yang, Fan Huang, Yihao Zhu, Jiayi Shi, Ling Pu, Geguang Dong, Jin Song Wang, Kailong |
| contents | Diffusion-based text-to-image (T2I) models enable high-quality image generation but also pose significant risks of misuse, particularly in producing not-safe-for-work (NSFW) content. While prior detection methods have focused on filtering prompts before generation or moderating images afterward, the in-generation phase of diffusion models remains largely unexplored for NSFW detection. In this paper, we introduce In-Generation Detection (IGD), a simple yet effective approach that leverages the predicted noise during the diffusion process as an internal signal to identify NSFW content. This approach is motivated by preliminary findings suggesting that the predicted noise may capture semantic cues that differentiate NSFW from benign prompts, even when the prompts are adversarially crafted. Experiments conducted on seven NSFW categories show that IGD achieves an average detection accuracy of 91.32% over naive and adversarial NSFW prompts, outperforming seven baseline methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03006 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models Yang, Fan Huang, Yihao Zhu, Jiayi Shi, Ling Pu, Geguang Dong, Jin Song Wang, Kailong Computer Vision and Pattern Recognition Diffusion-based text-to-image (T2I) models enable high-quality image generation but also pose significant risks of misuse, particularly in producing not-safe-for-work (NSFW) content. While prior detection methods have focused on filtering prompts before generation or moderating images afterward, the in-generation phase of diffusion models remains largely unexplored for NSFW detection. In this paper, we introduce In-Generation Detection (IGD), a simple yet effective approach that leverages the predicted noise during the diffusion process as an internal signal to identify NSFW content. This approach is motivated by preliminary findings suggesting that the predicted noise may capture semantic cues that differentiate NSFW from benign prompts, even when the prompts are adversarially crafted. Experiments conducted on seven NSFW categories show that IGD achieves an average detection accuracy of 91.32% over naive and adversarial NSFW prompts, outperforming seven baseline methods. |
| title | Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.03006 |