Task-Agnostic Attacks Against Vision Foundation Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916645024301056 |
|---|---|
| author | Pulfer, Brian Belousov, Yury Kinakh, Vitaliy Furon, Teddy Voloshynovskiy, Slava |
| author_facet | Pulfer, Brian Belousov, Yury Kinakh, Vitaliy Furon, Teddy Voloshynovskiy, Slava |
| contents | The study of security in machine learning mainly focuses on downstream task-specific attacks, where the adversarial example is obtained by optimizing a loss function specific to the downstream task. At the same time, it has become standard practice for machine learning practitioners to adopt publicly available pre-trained vision foundation models, effectively sharing a common backbone architecture across a multitude of applications such as classification, segmentation, depth estimation, retrieval, question-answering and more. The study of attacks on such foundation models and their impact to multiple downstream tasks remains vastly unexplored. This work proposes a general framework that forges task-agnostic adversarial examples by maximally disrupting the feature representation obtained with foundation models. We extensively evaluate the security of the feature representations obtained by popular vision foundation models by measuring the impact of this attack on multiple downstream tasks and its transferability between models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_03842 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Task-Agnostic Attacks Against Vision Foundation Models Pulfer, Brian Belousov, Yury Kinakh, Vitaliy Furon, Teddy Voloshynovskiy, Slava Computer Vision and Pattern Recognition Artificial Intelligence Cryptography and Security Machine Learning The study of security in machine learning mainly focuses on downstream task-specific attacks, where the adversarial example is obtained by optimizing a loss function specific to the downstream task. At the same time, it has become standard practice for machine learning practitioners to adopt publicly available pre-trained vision foundation models, effectively sharing a common backbone architecture across a multitude of applications such as classification, segmentation, depth estimation, retrieval, question-answering and more. The study of attacks on such foundation models and their impact to multiple downstream tasks remains vastly unexplored. This work proposes a general framework that forges task-agnostic adversarial examples by maximally disrupting the feature representation obtained with foundation models. We extensively evaluate the security of the feature representations obtained by popular vision foundation models by measuring the impact of this attack on multiple downstream tasks and its transferability between models. |
| title | Task-Agnostic Attacks Against Vision Foundation Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2503.03842 |