Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2603.27967 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910084372627456 |
|---|---|
| author | Jeong, Suchae Song, Jaehwi Lee, Haeone Kim, Hanna Kim, Jian Lee, Dongjun Shin, Dong Kyu Kim, Changyeon Hahm, Dongyoon Jin, Woogyeol Choi, Juheon Lee, Kimin |
| author_facet | Jeong, Suchae Song, Jaehwi Lee, Haeone Kim, Hanna Kim, Jian Lee, Dongjun Shin, Dong Kyu Kim, Changyeon Hahm, Dongyoon Jin, Woogyeol Choi, Juheon Lee, Kimin |
| contents | Vision-language models (VLMs) have achieved impressive results on single-view vision tasks, but lack the multi-view spatial reasoning capabilities essential for embodied AI systems to understand 3D environments and manipulate objects across different viewpoints. In this work, we introduce Cross-View Relations (XVR), a large-scale dataset designed to teach VLMs spatial reasoning across multiple views. XVR comprises 100K vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories, spanning three fundamental spatial reasoning tasks: Correspondence (matching objects across views), Verification (validating spatial relationships), and Localization (identifying object positions). VLMs fine-tuned on XVR achieve substantial improvements on established multi-view and robotic spatial reasoning benchmarks (MindCube and RoboSpatial). When integrated as backbones in Vision-Language-Action models, XVR-trained representations improve success rates on RoboCasa. Our results demonstrate that explicit training on cross-view spatial relations significantly enhances multi-view reasoning and transfers effectively to real-world robotic manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_27967 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning Multi-View Spatial Reasoning from Cross-View Relations Jeong, Suchae Song, Jaehwi Lee, Haeone Kim, Hanna Kim, Jian Lee, Dongjun Shin, Dong Kyu Kim, Changyeon Hahm, Dongyoon Jin, Woogyeol Choi, Juheon Lee, Kimin Computer Vision and Pattern Recognition Vision-language models (VLMs) have achieved impressive results on single-view vision tasks, but lack the multi-view spatial reasoning capabilities essential for embodied AI systems to understand 3D environments and manipulate objects across different viewpoints. In this work, we introduce Cross-View Relations (XVR), a large-scale dataset designed to teach VLMs spatial reasoning across multiple views. XVR comprises 100K vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories, spanning three fundamental spatial reasoning tasks: Correspondence (matching objects across views), Verification (validating spatial relationships), and Localization (identifying object positions). VLMs fine-tuned on XVR achieve substantial improvements on established multi-view and robotic spatial reasoning benchmarks (MindCube and RoboSpatial). When integrated as backbones in Vision-Language-Action models, XVR-trained representations improve success rates on RoboCasa. Our results demonstrate that explicit training on cross-view spatial relations significantly enhances multi-view reasoning and transfers effectively to real-world robotic manipulation. |
| title | Learning Multi-View Spatial Reasoning from Cross-View Relations |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.27967 |