Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910019785588736 |
|---|---|
| author | T V, Sethuraman Khosla, Savya Tiwari, Aditi Ganesh, Vidya Jayaprakash, Rakshana Jain, Aditya Srinivasakumar, Vignesh Susladkar, Onkar Kishor Sunkara, Srinidhi Shanmugham, Aditya Vaideeswaran, Rakesh Nishar, Abbaas Alif Mohamed Jenni, Simon Hoiem, Derek |
| author_facet | T V, Sethuraman Khosla, Savya Tiwari, Aditi Ganesh, Vidya Jayaprakash, Rakshana Jain, Aditya Srinivasakumar, Vignesh Susladkar, Onkar Kishor Sunkara, Srinidhi Shanmugham, Aditya Vaideeswaran, Rakesh Nishar, Abbaas Alif Mohamed Jenni, Simon Hoiem, Derek |
| contents | This work investigates a fundamental question: Do Video-Language Models (VidLMs) robustly account for video content, temporal sequence, and motion? Our investigation shows that, surprisingly, they often do not. We introduce REVEAL{}, a diagnostic benchmark that probes fundamental weaknesses of contemporary VidLMs through five controlled stress tests; assessing temporal expectation bias, reliance on language-only shortcuts, video sycophancy, camera motion sensitivity, and robustness to spatiotemporal occlusion. We test leading open- and closed-source VidLMs and find that these models confidently describe reversed scenes as forward, answer questions while neglecting video content, agree with false claims, struggle with basic camera motion, and fail to aggregate temporal information amidst simple spatiotemporal masking. Humans, on the other hand, succeed at these tasks with ease. Alongside our benchmark, we provide a data pipeline that automatically generates diagnostic examples for our stress tests, enabling broader and more scalable evaluation. We will release our benchmark and code to support future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_11244 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models T V, Sethuraman Khosla, Savya Tiwari, Aditi Ganesh, Vidya Jayaprakash, Rakshana Jain, Aditya Srinivasakumar, Vignesh Susladkar, Onkar Kishor Sunkara, Srinidhi Shanmugham, Aditya Vaideeswaran, Rakesh Nishar, Abbaas Alif Mohamed Jenni, Simon Hoiem, Derek Computer Vision and Pattern Recognition This work investigates a fundamental question: Do Video-Language Models (VidLMs) robustly account for video content, temporal sequence, and motion? Our investigation shows that, surprisingly, they often do not. We introduce REVEAL{}, a diagnostic benchmark that probes fundamental weaknesses of contemporary VidLMs through five controlled stress tests; assessing temporal expectation bias, reliance on language-only shortcuts, video sycophancy, camera motion sensitivity, and robustness to spatiotemporal occlusion. We test leading open- and closed-source VidLMs and find that these models confidently describe reversed scenes as forward, answer questions while neglecting video content, agree with false claims, struggle with basic camera motion, and fail to aggregate temporal information amidst simple spatiotemporal masking. Humans, on the other hand, succeed at these tasks with ease. Alongside our benchmark, we provide a data pipeline that automatically generates diagnostic examples for our stress tests, enabling broader and more scalable evaluation. We will release our benchmark and code to support future research. |
| title | Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.11244 |