SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914917141970944 |
|---|---|
| author | Han, Dongshen Zhang, Chaoning Zheng, Sheng Lu, Chang Yang, Yang Shen, Heng Tao |
| author_facet | Han, Dongshen Zhang, Chaoning Zheng, Sheng Lu, Chang Yang, Yang Shen, Heng Tao |
| contents | As Segment Anything Model (SAM) becomes a popular foundation model in computer vision, its adversarial robustness has become a concern that cannot be ignored. This works investigates whether it is possible to attack SAM with image-agnostic Universal Adversarial Perturbation (UAP). In other words, we seek a single perturbation that can fool the SAM to predict invalid masks for most (if not all) images. We demonstrate convetional image-centric attack framework is effective for image-independent attacks but fails for universal adversarial attack. To this end, we propose a novel perturbation-centric framework that results in a UAP generation method based on self-supervised contrastive learning (CL), where the UAP is set to the anchor sample and the positive sample is augmented from the UAP. The representations of negative samples are obtained from the image encoder in advance and saved in a memory bank. The effectiveness of our proposed CL-based UAP generation method is validated by both quantitative and qualitative results. On top of the ablation study to understand various components in our proposed method, we shed light on the roles of positive and negative samples in making the generated UAP effective for attacking SAM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_12431 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbation Han, Dongshen Zhang, Chaoning Zheng, Sheng Lu, Chang Yang, Yang Shen, Heng Tao Computer Vision and Pattern Recognition As Segment Anything Model (SAM) becomes a popular foundation model in computer vision, its adversarial robustness has become a concern that cannot be ignored. This works investigates whether it is possible to attack SAM with image-agnostic Universal Adversarial Perturbation (UAP). In other words, we seek a single perturbation that can fool the SAM to predict invalid masks for most (if not all) images. We demonstrate convetional image-centric attack framework is effective for image-independent attacks but fails for universal adversarial attack. To this end, we propose a novel perturbation-centric framework that results in a UAP generation method based on self-supervised contrastive learning (CL), where the UAP is set to the anchor sample and the positive sample is augmented from the UAP. The representations of negative samples are obtained from the image encoder in advance and saved in a memory bank. The effectiveness of our proposed CL-based UAP generation method is validated by both quantitative and qualitative results. On top of the ablation study to understand various components in our proposed method, we shed light on the roles of positive and negative samples in making the generated UAP effective for attacking SAM. |
| title | SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2310.12431 |