Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915593668526080 |
|---|---|
| author | Zhan, Xiaoyu Huang, Wenxuan Sun, Hao Fu, Xinyu Ma, Changfeng Cao, Shaosheng Jia, Bohan Lin, Shaohui Yin, Zhenfei Bai, Lei Ouyang, Wanli Li, Yuanqi Guo, Jie Guo, Yanwen |
| author_facet | Zhan, Xiaoyu Huang, Wenxuan Sun, Hao Fu, Xinyu Ma, Changfeng Cao, Shaosheng Jia, Bohan Lin, Shaohui Yin, Zhenfei Bai, Lei Ouyang, Wanli Li, Yuanqi Guo, Jie Guo, Yanwen |
| contents | Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tasks. However, it remains unclear whether these models can effectively capture the detailed spatial information required for robust real-world performance, especially cross-view consistency, a key requirement for accurate 3D reasoning. Considering this issue, we introduce Viewpoint Learning, a task designed to evaluate and improve the spatial reasoning capabilities of MLLMs. We present the Viewpoint-100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs. Our approach employs a two-stage fine-tuning strategy: first, foundational knowledge is injected to the baseline MLLM via Supervised Fine-Tuning (SFT) on Viewpoint-100K, resulting in significant improvements across multiple tasks; second, generalization is enhanced through Reinforcement Learning using the Group Relative Policy Optimization (GRPO) algorithm on a broader set of questions. Additionally, we introduce a hybrid cold-start initialization method designed to simultaneously learn viewpoint representations and maintain coherent reasoning thinking. Experimental results show that our approach significantly activates the spatial reasoning ability of MLLM, improving performance on both in-domain and out-of-domain reasoning tasks. Our findings highlight the value of developing foundational spatial skills in MLLMs, supporting future progress in robotics, autonomous systems, and 3D scene understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01618 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models Zhan, Xiaoyu Huang, Wenxuan Sun, Hao Fu, Xinyu Ma, Changfeng Cao, Shaosheng Jia, Bohan Lin, Shaohui Yin, Zhenfei Bai, Lei Ouyang, Wanli Li, Yuanqi Guo, Jie Guo, Yanwen Computer Vision and Pattern Recognition Computation and Language Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tasks. However, it remains unclear whether these models can effectively capture the detailed spatial information required for robust real-world performance, especially cross-view consistency, a key requirement for accurate 3D reasoning. Considering this issue, we introduce Viewpoint Learning, a task designed to evaluate and improve the spatial reasoning capabilities of MLLMs. We present the Viewpoint-100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs. Our approach employs a two-stage fine-tuning strategy: first, foundational knowledge is injected to the baseline MLLM via Supervised Fine-Tuning (SFT) on Viewpoint-100K, resulting in significant improvements across multiple tasks; second, generalization is enhanced through Reinforcement Learning using the Group Relative Policy Optimization (GRPO) algorithm on a broader set of questions. Additionally, we introduce a hybrid cold-start initialization method designed to simultaneously learn viewpoint representations and maintain coherent reasoning thinking. Experimental results show that our approach significantly activates the spatial reasoning ability of MLLM, improving performance on both in-domain and out-of-domain reasoning tasks. Our findings highlight the value of developing foundational spatial skills in MLLMs, supporting future progress in robotics, autonomous systems, and 3D scene understanding. |
| title | Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2511.01618 |