Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910205364666368 |
|---|---|
| author | Cao, Tri Le, Khoi Nguyen, Thong Nguyen, Cong-Duy Vo, Quynh Luu, Anh Tuan Miao, Chunyan Ng, See-Kiong Yan, Shuicheng Hooi, Bryan |
| author_facet | Cao, Tri Le, Khoi Nguyen, Thong Nguyen, Cong-Duy Vo, Quynh Luu, Anh Tuan Miao, Chunyan Ng, See-Kiong Yan, Shuicheng Hooi, Bryan |
| contents | While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We argue this stems from a failure in spatio-temporal monitoring, the ability to persistently track object identities, states, and relations over time. Existing benchmarks obscure this deficit by relying on single final-answer evaluations for queries that can often be resolved via local visual cues or statistical priors. To rigorously diagnose this, we introduce STEMO-Bench (Spatio-TEmporal MOnitoring), a benchmark of human-verified object-centric facts that evaluates intermediate reasoning by decomposing queries into sub-questions, distinguishing genuine temporal understanding from coincidental correctness. To address failure modes exposed by STEMO, we propose STEMO-Track, a novel object-centric framework that explicitly constructs and reasons over structured object trajectories via chunk-wise state extraction and temporal aggregation. Extensive experiments demonstrate that our object-centric framework significantly reduces hallucinated answers and improves spatio-temporal reasoning consistency over state-of-the-art MLLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_08974 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models Cao, Tri Le, Khoi Nguyen, Thong Nguyen, Cong-Duy Vo, Quynh Luu, Anh Tuan Miao, Chunyan Ng, See-Kiong Yan, Shuicheng Hooi, Bryan Computer Vision and Pattern Recognition Artificial Intelligence While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We argue this stems from a failure in spatio-temporal monitoring, the ability to persistently track object identities, states, and relations over time. Existing benchmarks obscure this deficit by relying on single final-answer evaluations for queries that can often be resolved via local visual cues or statistical priors. To rigorously diagnose this, we introduce STEMO-Bench (Spatio-TEmporal MOnitoring), a benchmark of human-verified object-centric facts that evaluates intermediate reasoning by decomposing queries into sub-questions, distinguishing genuine temporal understanding from coincidental correctness. To address failure modes exposed by STEMO, we propose STEMO-Track, a novel object-centric framework that explicitly constructs and reasons over structured object trajectories via chunk-wise state extraction and temporal aggregation. Extensive experiments demonstrate that our object-centric framework significantly reduces hallucinated answers and improves spatio-temporal reasoning consistency over state-of-the-art MLLMs. |
| title | Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2605.08974 |