Scaling Spatial Intelligence with Multimodal Foundation Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911549370662912 |
|---|---|
| author | Cai, Zhongang Wang, Ruisi Gu, Chenyang Pu, Fanyi Xu, Junxiang Wang, Yubo Yin, Wanqi Yang, Zhitao Wei, Chen Sun, Qingping Zhou, Tongxi Li, Jiaqi Pang, Hui En Qian, Oscar Wei, Yukun Lin, Zhiqian Shi, Xuanke Deng, Kewang Han, Xiaoyang Chen, Zukai Fan, Xiangyu Deng, Hanming Lu, Lewei Pan, Liang Li, Bo Liu, Ziwei Wang, Quan Lin, Dahua Yang, Lei |
| author_facet | Cai, Zhongang Wang, Ruisi Gu, Chenyang Pu, Fanyi Xu, Junxiang Wang, Yubo Yin, Wanqi Yang, Zhitao Wei, Chen Sun, Qingping Zhou, Tongxi Li, Jiaqi Pang, Hui En Qian, Oscar Wei, Yukun Lin, Zhiqian Shi, Xuanke Deng, Kewang Han, Xiaoyang Chen, Zukai Fan, Xiangyu Deng, Hanming Lu, Lewei Pan, Liang Li, Bo Liu, Ziwei Wang, Quan Lin, Dahua Yang, Lei |
| contents | Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models (i.e., Qwen3-VL and InternVL3) and unified understanding and generation models (i.e., Bagel). We take a principled approach to constructing high-performing and robust spatial intelligence by systematically curating SenseNova-SI-8M: eight million diverse data samples under a rigorous taxonomy of spatial capabilities. SenseNova-SI demonstrates unprecedented performance across a broad range of spatial intelligence benchmarks: 68.8% on VSI-Bench, 43.3% on MMSI, 85.7% on MindCube, 54.7% on ViewSpatial, 47.7% on SITE, 63.9% on BLINK, 55.5% on 3DSR, and 72.0% on EmbSpatial, while maintaining strong general multimodal understanding (e.g., 84.9% on MMBench-En). More importantly, we analyze the impact of data scaling, discuss early signs of emergent generalization capabilities enabled by diverse data training, analyze the risk of overfitting and language shortcuts, present a preliminary study on spatial chain-of-thought reasoning, and validate the potential downstream application. All newly trained multimodal foundation models are publicly released. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13719 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Spatial Intelligence with Multimodal Foundation Models Cai, Zhongang Wang, Ruisi Gu, Chenyang Pu, Fanyi Xu, Junxiang Wang, Yubo Yin, Wanqi Yang, Zhitao Wei, Chen Sun, Qingping Zhou, Tongxi Li, Jiaqi Pang, Hui En Qian, Oscar Wei, Yukun Lin, Zhiqian Shi, Xuanke Deng, Kewang Han, Xiaoyang Chen, Zukai Fan, Xiangyu Deng, Hanming Lu, Lewei Pan, Liang Li, Bo Liu, Ziwei Wang, Quan Lin, Dahua Yang, Lei Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Multimedia Robotics Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models (i.e., Qwen3-VL and InternVL3) and unified understanding and generation models (i.e., Bagel). We take a principled approach to constructing high-performing and robust spatial intelligence by systematically curating SenseNova-SI-8M: eight million diverse data samples under a rigorous taxonomy of spatial capabilities. SenseNova-SI demonstrates unprecedented performance across a broad range of spatial intelligence benchmarks: 68.8% on VSI-Bench, 43.3% on MMSI, 85.7% on MindCube, 54.7% on ViewSpatial, 47.7% on SITE, 63.9% on BLINK, 55.5% on 3DSR, and 72.0% on EmbSpatial, while maintaining strong general multimodal understanding (e.g., 84.9% on MMBench-En). More importantly, we analyze the impact of data scaling, discuss early signs of emergent generalization capabilities enabled by diverse data training, analyze the risk of overfitting and language shortcuts, present a preliminary study on spatial chain-of-thought reasoning, and validate the potential downstream application. All newly trained multimodal foundation models are publicly released. |
| title | Scaling Spatial Intelligence with Multimodal Foundation Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Multimedia Robotics |
| url | https://arxiv.org/abs/2511.13719 |