Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916626179293184 |
|---|---|
| author | Jun-Seong, Kim Kim, GeonU Yu-Ji, Kim Wang, Yu-Chiang Frank Choe, Jaesung Oh, Tae-Hyun |
| author_facet | Jun-Seong, Kim Kim, GeonU Yu-Ji, Kim Wang, Yu-Chiang Frank Choe, Jaesung Oh, Tae-Hyun |
| contents | We introduce Dr. Splat, a novel approach for open-vocabulary 3D scene understanding leveraging 3D Gaussian Splatting. Unlike existing language-embedded 3DGS methods, which rely on a rendering process, our method directly associates language-aligned CLIP embeddings with 3D Gaussians for holistic 3D scene understanding. The key of our method is a language feature registration technique where CLIP embeddings are assigned to the dominant Gaussians intersected by each pixel-ray. Moreover, we integrate Product Quantization (PQ) trained on general large-scale image data to compactly represent embeddings without per-scene optimization. Experiments demonstrate that our approach significantly outperforms existing approaches in 3D perception benchmarks, such as open-vocabulary 3D semantic segmentation, 3D object localization, and 3D object selection tasks. For video results, please visit : https://drsplat.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16652 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration Jun-Seong, Kim Kim, GeonU Yu-Ji, Kim Wang, Yu-Chiang Frank Choe, Jaesung Oh, Tae-Hyun Computer Vision and Pattern Recognition We introduce Dr. Splat, a novel approach for open-vocabulary 3D scene understanding leveraging 3D Gaussian Splatting. Unlike existing language-embedded 3DGS methods, which rely on a rendering process, our method directly associates language-aligned CLIP embeddings with 3D Gaussians for holistic 3D scene understanding. The key of our method is a language feature registration technique where CLIP embeddings are assigned to the dominant Gaussians intersected by each pixel-ray. Moreover, we integrate Product Quantization (PQ) trained on general large-scale image data to compactly represent embeddings without per-scene optimization. Experiments demonstrate that our approach significantly outperforms existing approaches in 3D perception benchmarks, such as open-vocabulary 3D semantic segmentation, 3D object localization, and 3D object selection tasks. For video results, please visit : https://drsplat.github.io/ |
| title | Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.16652 |