Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.05510 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911586520662016 |
|---|---|
| author | Xiu, Yanming Jiang, Zhengyuan Gong, Neil Zhenqiang Gorlatova, Maria |
| author_facet | Xiu, Yanming Jiang, Zhengyuan Gong, Neil Zhenqiang Gorlatova, Maria |
| contents | Augmented reality (AR) has rapidly expanded over the past decade. As AR becomes increasingly integrated into daily life, its security and reliability emerge as critical challenges. Among various threats, contradictory virtual content attacks, where malicious or inconsistent virtual elements are introduced into the user's view, pose a unique risk by misleading users, creating semantic confusion, or delivering harmful information. In this work, we systematically model such attacks and present ContrAR, a novel benchmark for evaluating the robustness of vision-language models (VLMs) against virtual content manipulation and contradiction in AR. ContrAR contains 312 real-world AR videos validated by 10 human participants. We further benchmark 11 VLMs, including both commercial and open-source models. Experimental results reveal that while current VLMs exhibit reasonable understanding of contradictory virtual content, room still remains for improvement in detecting and reasoning about adversarial content manipulations in AR environments. Moreover, balancing detection accuracy and latency remains challenging. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_05510 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality Xiu, Yanming Jiang, Zhengyuan Gong, Neil Zhenqiang Gorlatova, Maria Computer Vision and Pattern Recognition Augmented reality (AR) has rapidly expanded over the past decade. As AR becomes increasingly integrated into daily life, its security and reliability emerge as critical challenges. Among various threats, contradictory virtual content attacks, where malicious or inconsistent virtual elements are introduced into the user's view, pose a unique risk by misleading users, creating semantic confusion, or delivering harmful information. In this work, we systematically model such attacks and present ContrAR, a novel benchmark for evaluating the robustness of vision-language models (VLMs) against virtual content manipulation and contradiction in AR. ContrAR contains 312 real-world AR videos validated by 10 human participants. We further benchmark 11 VLMs, including both commercial and open-source models. Experimental results reveal that while current VLMs exhibit reasonable understanding of contradictory virtual content, room still remains for improvement in detecting and reasoning about adversarial content manipulations in AR environments. Moreover, balancing detection accuracy and latency remains challenging. |
| title | Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.05510 |