Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912948272758784 |
|---|---|
| author | Zhou, Keshen Chen, Runnan Gong, Mingming Liu, Tongliang |
| author_facet | Zhou, Keshen Chen, Runnan Gong, Mingming Liu, Tongliang |
| contents | Generalised 3D Referring Expression Segmentation (3D-GRES) localizes objects in 3D scenes based on natural language, even when descriptions match multiple or zero targets. Existing methods rely solely on sparse point clouds, lacking rich visual semantics for fine-grained descriptions. We propose HCF-RES, a multi-modal framework with two key innovations. First, Hierarchical Visual Semantic Decomposition leverages SAM instance masks to guide CLIP encoding at dual granularities -- pixel-level and instance-level features -- preserving object boundaries during 2D-to-3D projection. Second, Progressive Multi-level Fusion integrates representations through intra-modal collaboration, cross-modal adaptive weighting between 2D semantic and 3D geometric features, and language-guided refinement. HCF-RES achieves state-of-the-art results on both ScanRefer and Multi3DRefer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06250 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation Zhou, Keshen Chen, Runnan Gong, Mingming Liu, Tongliang Computer Vision and Pattern Recognition Generalised 3D Referring Expression Segmentation (3D-GRES) localizes objects in 3D scenes based on natural language, even when descriptions match multiple or zero targets. Existing methods rely solely on sparse point clouds, lacking rich visual semantics for fine-grained descriptions. We propose HCF-RES, a multi-modal framework with two key innovations. First, Hierarchical Visual Semantic Decomposition leverages SAM instance masks to guide CLIP encoding at dual granularities -- pixel-level and instance-level features -- preserving object boundaries during 2D-to-3D projection. Second, Progressive Multi-level Fusion integrates representations through intra-modal collaboration, cross-modal adaptive weighting between 2D semantic and 3D geometric features, and language-guided refinement. HCF-RES achieves state-of-the-art results on both ScanRefer and Multi3DRefer. |
| title | Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.06250 |