Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.01651 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929446193201152 |
|---|---|
| author | Zong, Yongshuo Yu, Tingyang Chavhan, Ruchika Zhao, Bingchen Hospedales, Timothy |
| author_facet | Zong, Yongshuo Yu, Tingyang Chavhan, Ruchika Zhao, Bingchen Hospedales, Timothy |
| contents | Large language and vision-language models are rapidly being deployed in practice thanks to their impressive capabilities in instruction following, in-context learning, and so on. This raises an urgent need to carefully analyse their robustness so that stakeholders can understand if and when such models are trustworthy enough to be relied upon in any given application. In this paper, we highlight a specific vulnerability in popular models, namely permutation sensitivity in multiple-choice question answering (MCQA). Specifically, we show empirically that popular models are vulnerable to adversarial permutation in answer sets for multiple-choice prompting, which is surprising as models should ideally be as invariant to prompt permutation as humans are. These vulnerabilities persist across various model sizes, and exist in very recent language and vision-language models. Code is available at https://github.com/ys-zong/FoolyourVLLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_01651 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations Zong, Yongshuo Yu, Tingyang Chavhan, Ruchika Zhao, Bingchen Hospedales, Timothy Machine Learning Large language and vision-language models are rapidly being deployed in practice thanks to their impressive capabilities in instruction following, in-context learning, and so on. This raises an urgent need to carefully analyse their robustness so that stakeholders can understand if and when such models are trustworthy enough to be relied upon in any given application. In this paper, we highlight a specific vulnerability in popular models, namely permutation sensitivity in multiple-choice question answering (MCQA). Specifically, we show empirically that popular models are vulnerable to adversarial permutation in answer sets for multiple-choice prompting, which is surprising as models should ideally be as invariant to prompt permutation as humans are. These vulnerabilities persist across various model sizes, and exist in very recent language and vision-language models. Code is available at https://github.com/ys-zong/FoolyourVLLMs. |
| title | Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2310.01651 |