Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916238469365760 |
|---|---|
| author | Thota, Poojitha Veerla, Jai Prakash Guttikonda, Partha Sai Nasr, Mohammad S. Nilizadeh, Shirin Luber, Jacob M. |
| author_facet | Thota, Poojitha Veerla, Jai Prakash Guttikonda, Partha Sai Nasr, Mohammad S. Nilizadeh, Shirin Luber, Jacob M. |
| contents | In the context of medical artificial intelligence, this study explores the vulnerabilities of the Pathology Language-Image Pretraining (PLIP) model, a Vision Language Foundation model, under targeted attacks. Leveraging the Kather Colon dataset with 7,180 H&E images across nine tissue types, our investigation employs Projected Gradient Descent (PGD) adversarial perturbation attacks to induce misclassifications intentionally. The outcomes reveal a 100% success rate in manipulating PLIP's predictions, underscoring its susceptibility to adversarial perturbations. The qualitative analysis of adversarial examples delves into the interpretability challenges, shedding light on nuanced changes in predictions induced by adversarial manipulations. These findings contribute crucial insights into the interpretability, domain adaptation, and trustworthiness of Vision Language Models in medical imaging. The study emphasizes the pressing need for robust defenses to ensure the reliability of AI models. The source codes for this experiment can be found at https://github.com/jaiprakash1824/VLM_Adv_Attack. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_02565 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging Thota, Poojitha Veerla, Jai Prakash Guttikonda, Partha Sai Nasr, Mohammad S. Nilizadeh, Shirin Luber, Jacob M. Image and Video Processing Computer Vision and Pattern Recognition Tissues and Organs In the context of medical artificial intelligence, this study explores the vulnerabilities of the Pathology Language-Image Pretraining (PLIP) model, a Vision Language Foundation model, under targeted attacks. Leveraging the Kather Colon dataset with 7,180 H&E images across nine tissue types, our investigation employs Projected Gradient Descent (PGD) adversarial perturbation attacks to induce misclassifications intentionally. The outcomes reveal a 100% success rate in manipulating PLIP's predictions, underscoring its susceptibility to adversarial perturbations. The qualitative analysis of adversarial examples delves into the interpretability challenges, shedding light on nuanced changes in predictions induced by adversarial manipulations. These findings contribute crucial insights into the interpretability, domain adaptation, and trustworthiness of Vision Language Models in medical imaging. The study emphasizes the pressing need for robust defenses to ensure the reliability of AI models. The source codes for this experiment can be found at https://github.com/jaiprakash1824/VLM_Adv_Attack. |
| title | Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Tissues and Organs |
| url | https://arxiv.org/abs/2401.02565 |