Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911297467056128 |
|---|---|
| author | Gong, Ziren Li, Xiaohan Tosi, Fabio Han, Jiawei Mattoccia, Stefano Cai, Jianfei Poggi, Matteo |
| author_facet | Gong, Ziren Li, Xiaohan Tosi, Fabio Han, Jiawei Mattoccia, Stefano Cai, Jianfei Poggi, Matteo |
| contents | We present Ov3R, a novel framework for open-vocabulary semantic 3D reconstruction from RGB video streams, designed to advance Spatial AI. The system features two key components: CLIP3R, a CLIP-informed 3D reconstruction module that predicts dense point maps from overlapping clips while embedding object-level semantics; and 2D-3D OVS, a 2D-3D open-vocabulary semantic module that lifts 2D features into 3D by learning fused descriptors integrating spatial, geometric, and semantic cues. Unlike prior methods, Ov3R incorporates CLIP semantics directly into the reconstruction process, enabling globally consistent geometry and fine-grained semantic alignment. Our framework achieves state-of-the-art performance in both dense 3D reconstruction and open-vocabulary 3D segmentation, marking a step forward toward real-time, semantics-aware Spatial AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_22052 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos Gong, Ziren Li, Xiaohan Tosi, Fabio Han, Jiawei Mattoccia, Stefano Cai, Jianfei Poggi, Matteo Computer Vision and Pattern Recognition We present Ov3R, a novel framework for open-vocabulary semantic 3D reconstruction from RGB video streams, designed to advance Spatial AI. The system features two key components: CLIP3R, a CLIP-informed 3D reconstruction module that predicts dense point maps from overlapping clips while embedding object-level semantics; and 2D-3D OVS, a 2D-3D open-vocabulary semantic module that lifts 2D features into 3D by learning fused descriptors integrating spatial, geometric, and semantic cues. Unlike prior methods, Ov3R incorporates CLIP semantics directly into the reconstruction process, enabling globally consistent geometry and fine-grained semantic alignment. Our framework achieves state-of-the-art performance in both dense 3D reconstruction and open-vocabulary 3D segmentation, marking a step forward toward real-time, semantics-aware Spatial AI. |
| title | Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.22052 |