Attacking Transformers with Feature Diversity Adversarial Perturbation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913263095119872 |
|---|---|
| author | Gao, Chenxing Zhou, Hang Yu, Junqing Ye, YuTeng Cai, Jiale Wang, Junle Yang, Wei |
| author_facet | Gao, Chenxing Zhou, Hang Yu, Junqing Ye, YuTeng Cai, Jiale Wang, Junle Yang, Wei |
| contents | Understanding the mechanisms behind Vision Transformer (ViT), particularly its vulnerability to adversarial perturba tions, is crucial for addressing challenges in its real-world applications. Existing ViT adversarial attackers rely on la bels to calculate the gradient for perturbation, and exhibit low transferability to other structures and tasks. In this paper, we present a label-free white-box attack approach for ViT-based models that exhibits strong transferability to various black box models, including most ViT variants, CNNs, and MLPs, even for models developed for other modalities. Our inspira tion comes from the feature collapse phenomenon in ViTs, where the critical attention mechanism overly depends on the low-frequency component of features, causing the features in middle-to-end layers to become increasingly similar and eventually collapse. We propose the feature diversity attacker to naturally accelerate this process and achieve remarkable performance and transferability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07942 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Attacking Transformers with Feature Diversity Adversarial Perturbation Gao, Chenxing Zhou, Hang Yu, Junqing Ye, YuTeng Cai, Jiale Wang, Junle Yang, Wei Cryptography and Security Computer Vision and Pattern Recognition Understanding the mechanisms behind Vision Transformer (ViT), particularly its vulnerability to adversarial perturba tions, is crucial for addressing challenges in its real-world applications. Existing ViT adversarial attackers rely on la bels to calculate the gradient for perturbation, and exhibit low transferability to other structures and tasks. In this paper, we present a label-free white-box attack approach for ViT-based models that exhibits strong transferability to various black box models, including most ViT variants, CNNs, and MLPs, even for models developed for other modalities. Our inspira tion comes from the feature collapse phenomenon in ViTs, where the critical attention mechanism overly depends on the low-frequency component of features, causing the features in middle-to-end layers to become increasingly similar and eventually collapse. We propose the feature diversity attacker to naturally accelerate this process and achieve remarkable performance and transferability. |
| title | Attacking Transformers with Feature Diversity Adversarial Perturbation |
| topic | Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.07942 |