Saved in:
Bibliographic Details
Main Authors: Gong, Ziren, Li, Xiaohan, Tosi, Fabio, Han, Jiawei, Mattoccia, Stefano, Cai, Jianfei, Poggi, Matteo
Format: Preprint
Published: 2025
Subjects:
Online Access:https://arxiv.org/abs/2507.22052
Tags: Add Tag
No Tags, Be the first to tag this record!
Table of Contents:
  • We present Ov3R, a novel framework for open-vocabulary semantic 3D reconstruction from RGB video streams, designed to advance Spatial AI. The system features two key components: CLIP3R, a CLIP-informed 3D reconstruction module that predicts dense point maps from overlapping clips while embedding object-level semantics; and 2D-3D OVS, a 2D-3D open-vocabulary semantic module that lifts 2D features into 3D by learning fused descriptors integrating spatial, geometric, and semantic cues. Unlike prior methods, Ov3R incorporates CLIP semantics directly into the reconstruction process, enabling globally consistent geometry and fine-grained semantic alignment. Our framework achieves state-of-the-art performance in both dense 3D reconstruction and open-vocabulary 3D segmentation, marking a step forward toward real-time, semantics-aware Spatial AI.