VideoMultiAgents: A Multi-Agent Framework for Video Question Answering
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915266658566144 |
|---|---|
| author | Kugo, Noriyuki Li, Xiang Li, Zixin Gupta, Ashish Khatua, Arpandeep Jain, Nidhish Patel, Chaitanya Kyuragi, Yuta Ishii, Yasunori Tanabiki, Masamoto Kozuka, Kazuki Adeli, Ehsan |
| author_facet | Kugo, Noriyuki Li, Xiang Li, Zixin Gupta, Ashish Khatua, Arpandeep Jain, Nidhish Patel, Chaitanya Kyuragi, Yuta Ishii, Yasunori Tanabiki, Masamoto Kozuka, Kazuki Adeli, Ehsan |
| contents | Video Question Answering (VQA) inherently relies on multimodal reasoning, integrating visual, temporal, and linguistic cues to achieve a deeper understanding of video content. However, many existing methods rely on feeding frame-level captions into a single model, making it difficult to adequately capture temporal and interactive contexts. To address this limitation, we introduce VideoMultiAgents, a framework that integrates specialized agents for vision, scene graph analysis, and text processing. It enhances video understanding leveraging complementary multimodal reasoning from independently operating agents. Our approach is also supplemented with a question-guided caption generation, which produces captions that highlight objects, actions, and temporal transitions directly relevant to a given query, thus improving the answer accuracy. Experimental results demonstrate that our method achieves state-of-the-art performance on Intent-QA (79.0%, +6.2% over previous SOTA), EgoSchema subset (75.4%, +3.4%), and NExT-QA (79.6%, +0.4%). The source code is available at https://github.com/PanasonicConnect/VideoMultiAgents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20091 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VideoMultiAgents: A Multi-Agent Framework for Video Question Answering Kugo, Noriyuki Li, Xiang Li, Zixin Gupta, Ashish Khatua, Arpandeep Jain, Nidhish Patel, Chaitanya Kyuragi, Yuta Ishii, Yasunori Tanabiki, Masamoto Kozuka, Kazuki Adeli, Ehsan Computer Vision and Pattern Recognition Multiagent Systems Video Question Answering (VQA) inherently relies on multimodal reasoning, integrating visual, temporal, and linguistic cues to achieve a deeper understanding of video content. However, many existing methods rely on feeding frame-level captions into a single model, making it difficult to adequately capture temporal and interactive contexts. To address this limitation, we introduce VideoMultiAgents, a framework that integrates specialized agents for vision, scene graph analysis, and text processing. It enhances video understanding leveraging complementary multimodal reasoning from independently operating agents. Our approach is also supplemented with a question-guided caption generation, which produces captions that highlight objects, actions, and temporal transitions directly relevant to a given query, thus improving the answer accuracy. Experimental results demonstrate that our method achieves state-of-the-art performance on Intent-QA (79.0%, +6.2% over previous SOTA), EgoSchema subset (75.4%, +3.4%), and NExT-QA (79.6%, +0.4%). The source code is available at https://github.com/PanasonicConnect/VideoMultiAgents. |
| title | VideoMultiAgents: A Multi-Agent Framework for Video Question Answering |
| topic | Computer Vision and Pattern Recognition Multiagent Systems |
| url | https://arxiv.org/abs/2504.20091 |