JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914387413958656 |
|---|---|
| author | Inuganti, Sandeep Kanayama, Hideaki Shimizu, Kanta Chamseddine, Mahdi Yokota, Soichiro Stricker, Didier Rambach, Jason |
| author_facet | Inuganti, Sandeep Kanayama, Hideaki Shimizu, Kanta Chamseddine, Mahdi Yokota, Soichiro Stricker, Didier Rambach, Jason |
| contents | Semantic segmentation across visual modalities such as 3D point clouds and panoramic images remains a challenging task, primarily due to the scarcity of annotated data and the limited adaptability of fixed-label models. In this paper, we present JOPP-3D, an open-vocabulary semantic segmentation framework that jointly leverages panoramic and point cloud data to enable language-driven scene understanding. We convert RGB-D panoramic images into their corresponding tangential perspective images and 3D point clouds, then use these modalities to extract and align foundational vision-language features. This allows natural language querying to generate semantic masks on both input modalities. Experimental evaluation on the Stanford-2D-3D-s and ToF-360 datasets demonstrates the capability of JOPP-3D to produce coherent and semantically meaningful segmentations across panoramic and 3D domains. Our proposed method achieves a significant improvement compared to the SOTA in open and closed vocabulary 2D and 3D semantic segmentation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06168 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas Inuganti, Sandeep Kanayama, Hideaki Shimizu, Kanta Chamseddine, Mahdi Yokota, Soichiro Stricker, Didier Rambach, Jason Computer Vision and Pattern Recognition Semantic segmentation across visual modalities such as 3D point clouds and panoramic images remains a challenging task, primarily due to the scarcity of annotated data and the limited adaptability of fixed-label models. In this paper, we present JOPP-3D, an open-vocabulary semantic segmentation framework that jointly leverages panoramic and point cloud data to enable language-driven scene understanding. We convert RGB-D panoramic images into their corresponding tangential perspective images and 3D point clouds, then use these modalities to extract and align foundational vision-language features. This allows natural language querying to generate semantic masks on both input modalities. Experimental evaluation on the Stanford-2D-3D-s and ToF-360 datasets demonstrates the capability of JOPP-3D to produce coherent and semantically meaningful segmentations across panoramic and 3D domains. Our proposed method achieves a significant improvement compared to the SOTA in open and closed vocabulary 2D and 3D semantic segmentation. |
| title | JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.06168 |