Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916878644936704 |
|---|---|
| author | Zhai, Shengfang Li, Jiajun Liu, Yue Chen, Huanran Tian, Zhihua Qu, Wenjie Shen, Qingni Jia, Ruoxi Dong, Yinpeng Zhang, Jiaheng |
| author_facet | Zhai, Shengfang Li, Jiajun Liu, Yue Chen, Huanran Tian, Zhihua Qu, Wenjie Shen, Qingni Jia, Ruoxi Dong, Yinpeng Zhang, Jiaheng |
| contents | In recent years, text-to-image (T2I) diffusion models have gained significant attention for their ability to generate high quality images reflecting text prompts. However, their growing popularity has also led to the emergence of backdoor threats, posing substantial risks. Currently, effective defense strategies against such threats are lacking due to the diversity of backdoor targets in T2I synthesis. In this paper, we propose NaviT2I, an efficient input-level backdoor defense framework against diverse T2I backdoors. Our approach is based on the new observation that trigger tokens tend to induce significant neuron activation variation in the early stage of the diffusion generation process, a phenomenon we term Early-step Activation Variation. Leveraging this insight, NaviT2I navigates T2I models to prevent malicious inputs by analyzing Neuron activation variations caused by input tokens. Extensive experiments show that NaviT2I significantly outperforms the baselines in both effectiveness and efficiency across diverse datasets, various T2I backdoors, and different model architectures including UNet and DiT. Furthermore, we show that our method remains effective under potential adaptive attacks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06453 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variation Zhai, Shengfang Li, Jiajun Liu, Yue Chen, Huanran Tian, Zhihua Qu, Wenjie Shen, Qingni Jia, Ruoxi Dong, Yinpeng Zhang, Jiaheng Cryptography and Security In recent years, text-to-image (T2I) diffusion models have gained significant attention for their ability to generate high quality images reflecting text prompts. However, their growing popularity has also led to the emergence of backdoor threats, posing substantial risks. Currently, effective defense strategies against such threats are lacking due to the diversity of backdoor targets in T2I synthesis. In this paper, we propose NaviT2I, an efficient input-level backdoor defense framework against diverse T2I backdoors. Our approach is based on the new observation that trigger tokens tend to induce significant neuron activation variation in the early stage of the diffusion generation process, a phenomenon we term Early-step Activation Variation. Leveraging this insight, NaviT2I navigates T2I models to prevent malicious inputs by analyzing Neuron activation variations caused by input tokens. Extensive experiments show that NaviT2I significantly outperforms the baselines in both effectiveness and efficiency across diverse datasets, various T2I backdoors, and different model architectures including UNet and DiT. Furthermore, we show that our method remains effective under potential adaptive attacks. |
| title | Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variation |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2503.06453 |