Video-in-the-Loop: Span-Grounded Long Video QA with Interleaved Reasoning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915537989140480 |
|---|---|
| author | Wang, Chendong Bai, Donglin Yang, Yifan Jin, Xiao Zhang, Anlan Wang, Rui Jiang, Shiqi Yang, Yuqing Wu, Hao Dai, Qi Luo, Chong Cao, Ting Qiu, Lili Banerjee, Suman |
| author_facet | Wang, Chendong Bai, Donglin Yang, Yifan Jin, Xiao Zhang, Anlan Wang, Rui Jiang, Shiqi Yang, Yuqing Wu, Hao Dai, Qi Luo, Chong Cao, Ting Qiu, Lili Banerjee, Suman |
| contents | We present \emph{Video-in-the-Loop} (ViTL), a two-stage long-video QA framework that preserves a fixed token budget by first \emph{localizing} question-relevant interval(s) with a low-fps skim and then \emph{answering} via span-aware reallocation of visual tokens at higher effective frame rate, emitting an interleaved output with both spans and the final option for direct attribution. We also introduce \dataname{}, which converts description based event graphs into \emph{span-grounded} multiple-choice QA by pairing each question with \emph{ground-truth} time span(s) and related reasoning. ViTL is trained end-to-end with an interleaved group-relative objective that couples temporal IoU for localization with answer correctness, allowing credit to flow from answers back to spans without increasing compute. Under fixed token budgets, ViTL attains up to 8.6% with 50% less frame input on long-video QA and temporal grounding (e.g., Charades-STA, ActivityNet-Captions) and ablations show that span-aware token reallocation consistently surpasses uniform sampling. Together, \dataname{} and ViTL provide an interpretable, compute-efficient recipe for scalable long-video QA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04022 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Video-in-the-Loop: Span-Grounded Long Video QA with Interleaved Reasoning Wang, Chendong Bai, Donglin Yang, Yifan Jin, Xiao Zhang, Anlan Wang, Rui Jiang, Shiqi Yang, Yuqing Wu, Hao Dai, Qi Luo, Chong Cao, Ting Qiu, Lili Banerjee, Suman Computer Vision and Pattern Recognition We present \emph{Video-in-the-Loop} (ViTL), a two-stage long-video QA framework that preserves a fixed token budget by first \emph{localizing} question-relevant interval(s) with a low-fps skim and then \emph{answering} via span-aware reallocation of visual tokens at higher effective frame rate, emitting an interleaved output with both spans and the final option for direct attribution. We also introduce \dataname{}, which converts description based event graphs into \emph{span-grounded} multiple-choice QA by pairing each question with \emph{ground-truth} time span(s) and related reasoning. ViTL is trained end-to-end with an interleaved group-relative objective that couples temporal IoU for localization with answer correctness, allowing credit to flow from answers back to spans without increasing compute. Under fixed token budgets, ViTL attains up to 8.6% with 50% less frame input on long-video QA and temporal grounding (e.g., Charades-STA, ActivityNet-Captions) and ablations show that span-aware token reallocation consistently surpasses uniform sampling. Together, \dataname{} and ViTL provide an interpretable, compute-efficient recipe for scalable long-video QA. |
| title | Video-in-the-Loop: Span-Grounded Long Video QA with Interleaved Reasoning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.04022 |