Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911040745242624 |
|---|---|
| author | Li, Yunxin Liu, Zhenyu Li, Zitao Zhang, Xuanyu Xu, Zhenran Chen, Xinyu Shi, Haoyuan Jiang, Shenyuan Wang, Xintong Wang, Jifang Huang, Shouzheng Zhao, Xinping Jiang, Borui Hong, Lanqing Wang, Longyue Tian, Zhuotao Huai, Baoxing Luo, Wenhan Luo, Weihua Zhang, Zheng Hu, Baotian Zhang, Min |
| author_facet | Li, Yunxin Liu, Zhenyu Li, Zitao Zhang, Xuanyu Xu, Zhenran Chen, Xinyu Shi, Haoyuan Jiang, Shenyuan Wang, Xintong Wang, Jifang Huang, Shouzheng Zhao, Xinping Jiang, Borui Hong, Lanqing Wang, Longyue Tian, Zhuotao Huai, Baoxing Luo, Wenhan Luo, Weihua Zhang, Zheng Hu, Baotian Zhang, Min |
| contents | Reasoning lies at the heart of intelligence, shaping the ability to make decisions, draw conclusions, and generalize across domains. In artificial intelligence, as systems increasingly operate in open, uncertain, and multimodal environments, reasoning becomes essential for enabling robust and adaptive behavior. Large Multimodal Reasoning Models (LMRMs) have emerged as a promising paradigm, integrating modalities such as text, images, audio, and video to support complex reasoning capabilities and aiming to achieve comprehensive perception, precise understanding, and deep reasoning. As research advances, multimodal reasoning has rapidly evolved from modular, perception-driven pipelines to unified, language-centric frameworks that offer more coherent cross-modal understanding. While instruction tuning and reinforcement learning have improved model reasoning, significant challenges remain in omni-modal generalization, reasoning depth, and agentic behavior. To address these issues, we present a comprehensive and structured survey of multimodal reasoning research, organized around a four-stage developmental roadmap that reflects the field's shifting design philosophies and emerging capabilities. First, we review early efforts based on task-specific modules, where reasoning was implicitly embedded across stages of representation, alignment, and fusion. Next, we examine recent approaches that unify reasoning into multimodal LLMs, with advances such as Multimodal Chain-of-Thought (MCoT) and multimodal reinforcement learning enabling richer and more structured reasoning chains. Finally, drawing on empirical insights from challenging benchmarks and experimental cases of OpenAI O3 and O4-mini, we discuss the conceptual direction of native large multimodal reasoning models (N-LMRMs), which aim to support scalable, agentic, and adaptive reasoning and planning in complex, real-world environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_04921 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models Li, Yunxin Liu, Zhenyu Li, Zitao Zhang, Xuanyu Xu, Zhenran Chen, Xinyu Shi, Haoyuan Jiang, Shenyuan Wang, Xintong Wang, Jifang Huang, Shouzheng Zhao, Xinping Jiang, Borui Hong, Lanqing Wang, Longyue Tian, Zhuotao Huai, Baoxing Luo, Wenhan Luo, Weihua Zhang, Zheng Hu, Baotian Zhang, Min Computer Vision and Pattern Recognition Computation and Language Reasoning lies at the heart of intelligence, shaping the ability to make decisions, draw conclusions, and generalize across domains. In artificial intelligence, as systems increasingly operate in open, uncertain, and multimodal environments, reasoning becomes essential for enabling robust and adaptive behavior. Large Multimodal Reasoning Models (LMRMs) have emerged as a promising paradigm, integrating modalities such as text, images, audio, and video to support complex reasoning capabilities and aiming to achieve comprehensive perception, precise understanding, and deep reasoning. As research advances, multimodal reasoning has rapidly evolved from modular, perception-driven pipelines to unified, language-centric frameworks that offer more coherent cross-modal understanding. While instruction tuning and reinforcement learning have improved model reasoning, significant challenges remain in omni-modal generalization, reasoning depth, and agentic behavior. To address these issues, we present a comprehensive and structured survey of multimodal reasoning research, organized around a four-stage developmental roadmap that reflects the field's shifting design philosophies and emerging capabilities. First, we review early efforts based on task-specific modules, where reasoning was implicitly embedded across stages of representation, alignment, and fusion. Next, we examine recent approaches that unify reasoning into multimodal LLMs, with advances such as Multimodal Chain-of-Thought (MCoT) and multimodal reinforcement learning enabling richer and more structured reasoning chains. Finally, drawing on empirical insights from challenging benchmarks and experimental cases of OpenAI O3 and O4-mini, we discuss the conceptual direction of native large multimodal reasoning models (N-LMRMs), which aim to support scalable, agentic, and adaptive reasoning and planning in complex, real-world environments. |
| title | Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2505.04921 |