Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917304516739072 |
|---|---|
| author | Feng, Zhiyuan Kang, Zhaolu Wang, Qijie Du, Zhiying Yan, Jiongrui Shi, Shubin Yuan, Chengbo Liang, Huizhi Deng, Yu Li, Qixiu Yang, Rushuai An, Arctanx Zheng, Leqi Wang, Weijie Chen, Shawn Xu, Sicheng Liang, Yaobo Yang, Jiaolong Guo, Baining |
| author_facet | Feng, Zhiyuan Kang, Zhaolu Wang, Qijie Du, Zhiying Yan, Jiongrui Shi, Shubin Yuan, Chengbo Liang, Huizhi Deng, Yu Li, Qixiu Yang, Rushuai An, Arctanx Zheng, Leqi Wang, Weijie Chen, Shawn Xu, Sicheng Liang, Yaobo Yang, Jiaolong Guo, Baining |
| contents | Vision-language models (VLMs) are essential to Embodied AI, enabling robots to perceive, reason, and act in complex environments. They also serve as the foundation for the recent Vision-Language-Action (VLA) models. Yet most evaluations of VLMs focus on single-view settings, leaving their ability to integrate multi-view information underexplored. At the same time, multi-camera setups are increasingly standard in robotic platforms, as they provide complementary perspectives to mitigate occlusion and depth ambiguity. Whether VLMs can effectively leverage such multi-view inputs for robotic reasoning therefore remains an open question. To bridge this gap, we introduce MV-RoboBench, a benchmark specifically designed to evaluate the multi-view spatial reasoning capabilities of VLMs in robotic manipulation. MV-RoboBench consists of 1.7k manually curated QA items across eight subtasks, divided into two primary categories: spatial understanding and robotic execution. We evaluate a diverse set of existing VLMs, including both open-source and closed-source models, along with enhanced versions incorporating CoT-inspired techniques. The results show that state-of-the-art models remain far below human performance, underscoring the substantial challenges VLMs face in multi-view robotic perception. Additionally, our analysis uncovers two key findings: (i) spatial intelligence and robotic task execution are positively correlated in multi-view robotic scenarios; and (ii) strong performance on existing general-purpose single-view spatial understanding benchmarks does not reliably translate to success in the robotic spatial tasks assessed by our benchmark. We release MV-RoboBench as an open resource to foster progress in spatially grounded VLMs and VLAs, providing not only data but also a standardized evaluation protocol for multi-view embodied reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_19400 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes Feng, Zhiyuan Kang, Zhaolu Wang, Qijie Du, Zhiying Yan, Jiongrui Shi, Shubin Yuan, Chengbo Liang, Huizhi Deng, Yu Li, Qixiu Yang, Rushuai An, Arctanx Zheng, Leqi Wang, Weijie Chen, Shawn Xu, Sicheng Liang, Yaobo Yang, Jiaolong Guo, Baining Computer Vision and Pattern Recognition Vision-language models (VLMs) are essential to Embodied AI, enabling robots to perceive, reason, and act in complex environments. They also serve as the foundation for the recent Vision-Language-Action (VLA) models. Yet most evaluations of VLMs focus on single-view settings, leaving their ability to integrate multi-view information underexplored. At the same time, multi-camera setups are increasingly standard in robotic platforms, as they provide complementary perspectives to mitigate occlusion and depth ambiguity. Whether VLMs can effectively leverage such multi-view inputs for robotic reasoning therefore remains an open question. To bridge this gap, we introduce MV-RoboBench, a benchmark specifically designed to evaluate the multi-view spatial reasoning capabilities of VLMs in robotic manipulation. MV-RoboBench consists of 1.7k manually curated QA items across eight subtasks, divided into two primary categories: spatial understanding and robotic execution. We evaluate a diverse set of existing VLMs, including both open-source and closed-source models, along with enhanced versions incorporating CoT-inspired techniques. The results show that state-of-the-art models remain far below human performance, underscoring the substantial challenges VLMs face in multi-view robotic perception. Additionally, our analysis uncovers two key findings: (i) spatial intelligence and robotic task execution are positively correlated in multi-view robotic scenarios; and (ii) strong performance on existing general-purpose single-view spatial understanding benchmarks does not reliably translate to success in the robotic spatial tasks assessed by our benchmark. We release MV-RoboBench as an open resource to foster progress in spatially grounded VLMs and VLAs, providing not only data but also a standardized evaluation protocol for multi-view embodied reasoning. |
| title | Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.19400 |