OpenFusion++: An Open-vocabulary Real-time Scene Understanding System
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918001596432384 |
|---|---|
| author | Jin, Xiaofeng Frosi, Matteo Matteucci, Matteo |
| author_facet | Jin, Xiaofeng Frosi, Matteo Matteucci, Matteo |
| contents | Real-time open-vocabulary scene understanding is essential for efficient 3D perception in applications such as vision-language navigation, embodied intelligence, and augmented reality. However, existing methods suffer from imprecise instance segmentation, static semantic updates, and limited handling of complex queries. To address these issues, we present OpenFusion++, a TSDF-based real-time 3D semantic-geometric reconstruction system. Our approach refines 3D point clouds by fusing confidence maps from foundational models, dynamically updates global semantic labels via an adaptive cache based on instance area, and employs a dual-path encoding framework that integrates object attributes with environmental context for precise query responses. Experiments on the ICL, Replica, ScanNet, and ScanNet++ datasets demonstrate that OpenFusion++ significantly outperforms the baseline in both semantic accuracy and query responsiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_19266 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OpenFusion++: An Open-vocabulary Real-time Scene Understanding System Jin, Xiaofeng Frosi, Matteo Matteucci, Matteo Computer Vision and Pattern Recognition 68T45, 68U05 I.2.10; I.4.8 Real-time open-vocabulary scene understanding is essential for efficient 3D perception in applications such as vision-language navigation, embodied intelligence, and augmented reality. However, existing methods suffer from imprecise instance segmentation, static semantic updates, and limited handling of complex queries. To address these issues, we present OpenFusion++, a TSDF-based real-time 3D semantic-geometric reconstruction system. Our approach refines 3D point clouds by fusing confidence maps from foundational models, dynamically updates global semantic labels via an adaptive cache based on instance area, and employs a dual-path encoding framework that integrates object attributes with environmental context for precise query responses. Experiments on the ICL, Replica, ScanNet, and ScanNet++ datasets demonstrate that OpenFusion++ significantly outperforms the baseline in both semantic accuracy and query responsiveness. |
| title | OpenFusion++: An Open-vocabulary Real-time Scene Understanding System |
| topic | Computer Vision and Pattern Recognition 68T45, 68U05 I.2.10; I.4.8 |
| url | https://arxiv.org/abs/2504.19266 |