Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913756690251776 |
|---|---|
| author | Zhang, Chenyangguang Delitzas, Alexandros Wang, Fangjinhua Zhang, Ruida Ji, Xiangyang Pollefeys, Marc Engelmann, Francis |
| author_facet | Zhang, Chenyangguang Delitzas, Alexandros Wang, Fangjinhua Zhang, Ruida Ji, Xiangyang Pollefeys, Marc Engelmann, Francis |
| contents | We introduce the task of predicting functional 3D scene graphs for real-world indoor environments from posed RGB-D images. Unlike traditional 3D scene graphs that focus on spatial relationships of objects, functional 3D scene graphs capture objects, interactive elements, and their functional relationships. Due to the lack of training data, we leverage foundation models, including visual language models (VLMs) and large language models (LLMs), to encode functional knowledge. We evaluate our approach on an extended SceneFun3D dataset and a newly collected dataset, FunGraph3D, both annotated with functional 3D scene graphs. Our method significantly outperforms adapted baselines, including Open3DSG and ConceptGraph, demonstrating its effectiveness in modeling complex scene functionalities. We also demonstrate downstream applications such as 3D question answering and robotic manipulation using functional 3D scene graphs. See our project page at https://openfungraph.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_19199 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces Zhang, Chenyangguang Delitzas, Alexandros Wang, Fangjinhua Zhang, Ruida Ji, Xiangyang Pollefeys, Marc Engelmann, Francis Computer Vision and Pattern Recognition We introduce the task of predicting functional 3D scene graphs for real-world indoor environments from posed RGB-D images. Unlike traditional 3D scene graphs that focus on spatial relationships of objects, functional 3D scene graphs capture objects, interactive elements, and their functional relationships. Due to the lack of training data, we leverage foundation models, including visual language models (VLMs) and large language models (LLMs), to encode functional knowledge. We evaluate our approach on an extended SceneFun3D dataset and a newly collected dataset, FunGraph3D, both annotated with functional 3D scene graphs. Our method significantly outperforms adapted baselines, including Open3DSG and ConceptGraph, demonstrating its effectiveness in modeling complex scene functionalities. We also demonstrate downstream applications such as 3D question answering and robotic manipulation using functional 3D scene graphs. See our project page at https://openfungraph.github.io |
| title | Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.19199 |