Backdoor Defense in Diffusion Models via Spatial Attention Unlearning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910919260372992 |
|---|---|
| author | Jha, Abha Aravindan, Ashwath Vaithinathan Salaway, Matthew Bhide, Atharva Sandeep Yaldiz, Duygu Nur |
| author_facet | Jha, Abha Aravindan, Ashwath Vaithinathan Salaway, Matthew Bhide, Atharva Sandeep Yaldiz, Duygu Nur |
| contents | Text-to-image diffusion models are increasingly vulnerable to backdoor attacks, where malicious modifications to the training data cause the model to generate unintended outputs when specific triggers are present. While classification models have seen extensive development of defense mechanisms, generative models remain largely unprotected due to their high-dimensional output space, which complicates the detection and mitigation of subtle perturbations. Defense strategies for diffusion models, in particular, remain under-explored. In this work, we propose Spatial Attention Unlearning (SAU), a novel technique for mitigating backdoor attacks in diffusion models. SAU leverages latent space manipulation and spatial attention mechanisms to isolate and remove the latent representation of backdoor triggers, ensuring precise and efficient removal of malicious effects. We evaluate SAU across various types of backdoor attacks, including pixel-based and style-based triggers, and demonstrate its effectiveness in achieving 100% trigger removal accuracy. Furthermore, SAU achieves a CLIP score of 0.7023, outperforming existing methods while preserving the model's ability to generate high-quality, semantically aligned images. Our results show that SAU is a robust, scalable, and practical solution for securing text-to-image diffusion models against backdoor attacks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18563 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Backdoor Defense in Diffusion Models via Spatial Attention Unlearning Jha, Abha Aravindan, Ashwath Vaithinathan Salaway, Matthew Bhide, Atharva Sandeep Yaldiz, Duygu Nur Cryptography and Security Artificial Intelligence Computer Vision and Pattern Recognition Text-to-image diffusion models are increasingly vulnerable to backdoor attacks, where malicious modifications to the training data cause the model to generate unintended outputs when specific triggers are present. While classification models have seen extensive development of defense mechanisms, generative models remain largely unprotected due to their high-dimensional output space, which complicates the detection and mitigation of subtle perturbations. Defense strategies for diffusion models, in particular, remain under-explored. In this work, we propose Spatial Attention Unlearning (SAU), a novel technique for mitigating backdoor attacks in diffusion models. SAU leverages latent space manipulation and spatial attention mechanisms to isolate and remove the latent representation of backdoor triggers, ensuring precise and efficient removal of malicious effects. We evaluate SAU across various types of backdoor attacks, including pixel-based and style-based triggers, and demonstrate its effectiveness in achieving 100% trigger removal accuracy. Furthermore, SAU achieves a CLIP score of 0.7023, outperforming existing methods while preserving the model's ability to generate high-quality, semantically aligned images. Our results show that SAU is a robust, scalable, and practical solution for securing text-to-image diffusion models against backdoor attacks. |
| title | Backdoor Defense in Diffusion Models via Spatial Attention Unlearning |
| topic | Cryptography and Security Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.18563 |