Grounded 3D-Aware Spatial Vision-Language Modeling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918530074542080 |
|---|---|
| author | Cheng, An-Chieh Fu, Yang Ji, Yatai Zhu, Ligeng Zhan, Guanqi Zhang, Zhuoyang Yang, Zhaojing Han, Song Lu, Yao Molchanov, Pavlo Murali, Vidya Nariyambut Kautz, Jan Wang, Xiaolong Yin, Hongxu Liu, Sifei |
| author_facet | Cheng, An-Chieh Fu, Yang Ji, Yatai Zhu, Ligeng Zhan, Guanqi Zhang, Zhuoyang Yang, Zhaojing Han, Song Lu, Yao Molchanov, Pavlo Murali, Vidya Nariyambut Kautz, Jan Wang, Xiaolong Yin, Hongxu Liu, Sifei |
| contents | We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding--within a single framework. GR3D introduces an implicit grounding mechanism that identifies entity mentions during generation and inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly when producing spatial chain-of-thought responses. In parallel, a region-prompted monocular 3D grounding design predicts 3D bounding boxes in the camera view from grounded region queries, supported by intrinsic-aware normalization and dense geometric supervision. Together, these grounding capabilities enable GR3D to decompose complex spatial understanding problems into grounded 2D perception followed by 3D inference. GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs. These grounding capabilities collectively enhance general spatial understanding beyond the grounding task itself. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_30307 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Grounded 3D-Aware Spatial Vision-Language Modeling Cheng, An-Chieh Fu, Yang Ji, Yatai Zhu, Ligeng Zhan, Guanqi Zhang, Zhuoyang Yang, Zhaojing Han, Song Lu, Yao Molchanov, Pavlo Murali, Vidya Nariyambut Kautz, Jan Wang, Xiaolong Yin, Hongxu Liu, Sifei Computer Vision and Pattern Recognition We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding--within a single framework. GR3D introduces an implicit grounding mechanism that identifies entity mentions during generation and inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly when producing spatial chain-of-thought responses. In parallel, a region-prompted monocular 3D grounding design predicts 3D bounding boxes in the camera view from grounded region queries, supported by intrinsic-aware normalization and dense geometric supervision. Together, these grounding capabilities enable GR3D to decompose complex spatial understanding problems into grounded 2D perception followed by 3D inference. GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs. These grounding capabilities collectively enhance general spatial understanding beyond the grounding task itself. |
| title | Grounded 3D-Aware Spatial Vision-Language Modeling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.30307 |