SegRGB-X: General RGB-X Semantic Segmentation Model
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911552897024000 |
|---|---|
| author | Liu, Jiong Xu, Yingjie Zhou, Xingcheng Song, Rui Zimmer, Walter Knoll, Alois Cao, Hu |
| author_facet | Liu, Jiong Xu, Yingjie Zhou, Xingcheng Song, Rui Zimmer, Walter Knoll, Alois Cao, Hu |
| contents | Semantic segmentation across arbitrary sensor modalities faces significant challenges due to diverse sensor characteristics, and the traditional configurations for this task result in redundant development efforts. We address these challenges by introducing a universal arbitrary-modal semantic segmentation framework that unifies segmentation across multiple modalities. Our approach features three key innovations: (1) the Modality-aware CLIP (MA-CLIP), which provides modality-specific scene understanding guidance through LoRA fine-tuning; (2) Modality-aligned Embeddings for capturing fine-grained features; and (3) the Domain-specific Refinement Module (DSRM) for dynamic feature adjustment. Evaluated on five diverse datasets with different complementary modalities (event, thermal, depth, polarization, and light field), our model surpasses specialized multi-modal methods and achieves state-of-the-art performance with a mIoU of 65.03%. The codes will be released upon acceptance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_28023 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SegRGB-X: General RGB-X Semantic Segmentation Model Liu, Jiong Xu, Yingjie Zhou, Xingcheng Song, Rui Zimmer, Walter Knoll, Alois Cao, Hu Computer Vision and Pattern Recognition Semantic segmentation across arbitrary sensor modalities faces significant challenges due to diverse sensor characteristics, and the traditional configurations for this task result in redundant development efforts. We address these challenges by introducing a universal arbitrary-modal semantic segmentation framework that unifies segmentation across multiple modalities. Our approach features three key innovations: (1) the Modality-aware CLIP (MA-CLIP), which provides modality-specific scene understanding guidance through LoRA fine-tuning; (2) Modality-aligned Embeddings for capturing fine-grained features; and (3) the Domain-specific Refinement Module (DSRM) for dynamic feature adjustment. Evaluated on five diverse datasets with different complementary modalities (event, thermal, depth, polarization, and light field), our model surpasses specialized multi-modal methods and achieves state-of-the-art performance with a mIoU of 65.03%. The codes will be released upon acceptance. |
| title | SegRGB-X: General RGB-X Semantic Segmentation Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.28023 |