Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.05079 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913009309319168 |
|---|---|
| author | Yang, Zhongyu Yang, Zuhao Zhan, Shuo Yue, Tan Pang, Wei Yuan, Yingfang |
| author_facet | Yang, Zhongyu Yang, Zuhao Zhan, Shuo Yue, Tan Pang, Wei Yuan, Yingfang |
| contents | Video question answering (VideoQA) is a challenging task that requires integrating spatial, temporal, and semantic information to capture the complex dynamics of video sequences. Although recent advances have introduced various approaches for video understanding, most existing methods still rely on locating relevant frames to answer questions rather than reasoning through the evolving storyline as humans do. Humans naturally interpret videos through coherent storylines, an ability that is crucial for making robust and contextually grounded predictions. To address this gap, we propose SVAgent, a storyline-guided cross-modal multi-agent framework for VideoQA. The storyline agent progressively constructs a narrative representation based on frames suggested by a refinement suggestion agent that analyzes historical failures. In addition, cross-modal decision agents independently predict answers from visual and textual modalities under the guidance of the evolving storyline. Their outputs are then evaluated by a meta-agent to align cross-modal predictions and enhance reasoning robustness and answer consistency. Experimental results demonstrate that SVAgent achieves superior performance and interpretability by emulating human-like storyline reasoning in video understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_05079 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaboration Yang, Zhongyu Yang, Zuhao Zhan, Shuo Yue, Tan Pang, Wei Yuan, Yingfang Computer Vision and Pattern Recognition Video question answering (VideoQA) is a challenging task that requires integrating spatial, temporal, and semantic information to capture the complex dynamics of video sequences. Although recent advances have introduced various approaches for video understanding, most existing methods still rely on locating relevant frames to answer questions rather than reasoning through the evolving storyline as humans do. Humans naturally interpret videos through coherent storylines, an ability that is crucial for making robust and contextually grounded predictions. To address this gap, we propose SVAgent, a storyline-guided cross-modal multi-agent framework for VideoQA. The storyline agent progressively constructs a narrative representation based on frames suggested by a refinement suggestion agent that analyzes historical failures. In addition, cross-modal decision agents independently predict answers from visual and textual modalities under the guidance of the evolving storyline. Their outputs are then evaluated by a meta-agent to align cross-modal predictions and enhance reasoning robustness and answer consistency. Experimental results demonstrate that SVAgent achieves superior performance and interpretability by emulating human-like storyline reasoning in video understanding. |
| title | SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaboration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.05079 |