STARS: Shared-specific Translation and Alignment for missing-modality Remote Sensing Semantic Segmentation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911396240818176 |
|---|---|
| author | Wang, Tong Zhang, Xiaodong Chen, Guanzhou Wang, Jiaqi Liu, Chenxi Tan, Xiaoliang Guo, Wenchao Li, Xuyang Wang, Xuanrui Wang, Zifan |
| author_facet | Wang, Tong Zhang, Xiaodong Chen, Guanzhou Wang, Jiaqi Liu, Chenxi Tan, Xiaoliang Guo, Wenchao Li, Xuyang Wang, Xuanrui Wang, Zifan |
| contents | Multimodal remote sensing technology significantly enhances the understanding of surface semantics by integrating heterogeneous data such as optical images, Synthetic Aperture Radar (SAR), and Digital Surface Models (DSM). However, in practical applications, the missing of modality data (e.g., optical or DSM) is a common and severe challenge, which leads to performance decline in traditional multimodal fusion models. Existing methods for addressing missing modalities still face limitations, including feature collapse and overly generalized recovered features. To address these issues, we propose \textbf{STARS} (\textbf{S}hared-specific \textbf{T}ranslation and \textbf{A}lignment for missing-modality \textbf{R}emote \textbf{S}ensing), a robust semantic segmentation framework for incomplete multimodal inputs. STARS is built on two key designs. First, we introduce an asymmetric alignment mechanism with bidirectional translation and stop-gradient, which effectively prevents feature collapse and reduces sensitivity to hyperparameters. Second, we propose a Pixel-level Semantic sampling Alignment (PSA) strategy that combines class-balanced pixel sampling with cross-modality semantic alignment loss, to mitigate alignment failures caused by severe class imbalance and improve minority-class recognition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_17342 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | STARS: Shared-specific Translation and Alignment for missing-modality Remote Sensing Semantic Segmentation Wang, Tong Zhang, Xiaodong Chen, Guanzhou Wang, Jiaqi Liu, Chenxi Tan, Xiaoliang Guo, Wenchao Li, Xuyang Wang, Xuanrui Wang, Zifan Computer Vision and Pattern Recognition Multimodal remote sensing technology significantly enhances the understanding of surface semantics by integrating heterogeneous data such as optical images, Synthetic Aperture Radar (SAR), and Digital Surface Models (DSM). However, in practical applications, the missing of modality data (e.g., optical or DSM) is a common and severe challenge, which leads to performance decline in traditional multimodal fusion models. Existing methods for addressing missing modalities still face limitations, including feature collapse and overly generalized recovered features. To address these issues, we propose \textbf{STARS} (\textbf{S}hared-specific \textbf{T}ranslation and \textbf{A}lignment for missing-modality \textbf{R}emote \textbf{S}ensing), a robust semantic segmentation framework for incomplete multimodal inputs. STARS is built on two key designs. First, we introduce an asymmetric alignment mechanism with bidirectional translation and stop-gradient, which effectively prevents feature collapse and reduces sensitivity to hyperparameters. Second, we propose a Pixel-level Semantic sampling Alignment (PSA) strategy that combines class-balanced pixel sampling with cross-modality semantic alignment loss, to mitigate alignment failures caused by severe class imbalance and improve minority-class recognition. |
| title | STARS: Shared-specific Translation and Alignment for missing-modality Remote Sensing Semantic Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.17342 |