Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910106418937856 |
|---|---|
| author | Tong, Jingqi Mou, Yurong Li, Hangcheng Li, Mingzhe Yang, Yongzhuo Zhang, Ming Chen, Qiguang Liang, Tianyi Hu, Xiaomeng Zheng, Yining Chen, Xinchi Zhao, Jun Huang, Xuanjing Qiu, Xipeng |
| author_facet | Tong, Jingqi Mou, Yurong Li, Hangcheng Li, Mingzhe Yang, Yongzhuo Zhang, Ming Chen, Qiguang Liang, Tianyi Hu, Xiaomeng Zheng, Yining Chen, Xinchi Zhao, Jun Huang, Xuanjing Qiu, Xipeng |
| contents | The "Thinking with Text" and "Thinking with Images" paradigms significantly improve the reasoning abilities of large language models (LLMs) and Vision-Language Models (VLMs). However, these paradigms have inherent limitations. (1) Images capture only single moments and fail to represent dynamic processes or continuous changes, and (2) The separation of text and vision as distinct modalities, which hinders unified multimodal understanding and generation. Therefore, we propose "Thinking with Video", a new paradigm that leverages video generation models such as Sora-2 to use video frames as a unified medium for multimodal reasoning. To support this exploration, we developed the Video Thinking Benchmark (VideoThinkBench), which covers both vision-centric tasks (e.g., Eyeballing Puzzles) and text-centric tasks (e.g., GSM8K and MMMU). Our evaluation on VideoThinkBench establishes Sora-2 as a capable reasoner. On vision-centric tasks, Sora-2 is comparable to state-of-the-art (SOTA) VLMs, and even surpasses GPT-5 by 10% on eyeballing puzzles. On text-centric tasks, Sora-2 achieves 92% accuracy on MATH, and 69.2% accuracy on MMMU. Furthermore, we systematically analyze the source of these abilities. We also find that self-consistency and in-context learning can improve Sora-2's performance. In summary, our findings show that the video generation model is the potential unified multimodal understanding and generation model, positioning "Thinking with Video" as a potential unified multimodal reasoning paradigm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04570 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm Tong, Jingqi Mou, Yurong Li, Hangcheng Li, Mingzhe Yang, Yongzhuo Zhang, Ming Chen, Qiguang Liang, Tianyi Hu, Xiaomeng Zheng, Yining Chen, Xinchi Zhao, Jun Huang, Xuanjing Qiu, Xipeng Computer Vision and Pattern Recognition Computation and Language The "Thinking with Text" and "Thinking with Images" paradigms significantly improve the reasoning abilities of large language models (LLMs) and Vision-Language Models (VLMs). However, these paradigms have inherent limitations. (1) Images capture only single moments and fail to represent dynamic processes or continuous changes, and (2) The separation of text and vision as distinct modalities, which hinders unified multimodal understanding and generation. Therefore, we propose "Thinking with Video", a new paradigm that leverages video generation models such as Sora-2 to use video frames as a unified medium for multimodal reasoning. To support this exploration, we developed the Video Thinking Benchmark (VideoThinkBench), which covers both vision-centric tasks (e.g., Eyeballing Puzzles) and text-centric tasks (e.g., GSM8K and MMMU). Our evaluation on VideoThinkBench establishes Sora-2 as a capable reasoner. On vision-centric tasks, Sora-2 is comparable to state-of-the-art (SOTA) VLMs, and even surpasses GPT-5 by 10% on eyeballing puzzles. On text-centric tasks, Sora-2 achieves 92% accuracy on MATH, and 69.2% accuracy on MMMU. Furthermore, we systematically analyze the source of these abilities. We also find that self-consistency and in-context learning can improve Sora-2's performance. In summary, our findings show that the video generation model is the potential unified multimodal understanding and generation model, positioning "Thinking with Video" as a potential unified multimodal reasoning paradigm. |
| title | Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2511.04570 |