Multi-view Remote Sensing Image Segmentation With SAM priors
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913359680503808 |
|---|---|
| author | Qi, Zipeng Liu, Chenyang Liu, Zili Chen, Hao Wu, Yongchang Zou, Zhengxia Sh, Zhenwei |
| author_facet | Qi, Zipeng Liu, Chenyang Liu, Zili Chen, Hao Wu, Yongchang Zou, Zhengxia Sh, Zhenwei |
| contents | Multi-view segmentation in Remote Sensing (RS) seeks to segment images from diverse perspectives within a scene. Recent methods leverage 3D information extracted from an Implicit Neural Field (INF), bolstering result consistency across multiple views while using limited accounts of labels (even within 3-5 labels) to streamline labor. Nonetheless, achieving superior performance within the constraints of limited-view labels remains challenging due to inadequate scene-wide supervision and insufficient semantic features within the INF. To address these. we propose to inject the prior of the visual foundation model-Segment Anything(SAM), to the INF to obtain better results under the limited number of training data. Specifically, we contrast SAM features between testing and training views to derive pseudo labels for each testing view, augmenting scene-wide labeling information. Subsequently, we introduce SAM features via a transformer into the INF of the scene, supplementing the semantic information. The experimental results demonstrate that our method outperforms the mainstream method, confirming the efficacy of SAM as a supplement to the INF for this task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_14171 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-view Remote Sensing Image Segmentation With SAM priors Qi, Zipeng Liu, Chenyang Liu, Zili Chen, Hao Wu, Yongchang Zou, Zhengxia Sh, Zhenwei Computer Vision and Pattern Recognition Multi-view segmentation in Remote Sensing (RS) seeks to segment images from diverse perspectives within a scene. Recent methods leverage 3D information extracted from an Implicit Neural Field (INF), bolstering result consistency across multiple views while using limited accounts of labels (even within 3-5 labels) to streamline labor. Nonetheless, achieving superior performance within the constraints of limited-view labels remains challenging due to inadequate scene-wide supervision and insufficient semantic features within the INF. To address these. we propose to inject the prior of the visual foundation model-Segment Anything(SAM), to the INF to obtain better results under the limited number of training data. Specifically, we contrast SAM features between testing and training views to derive pseudo labels for each testing view, augmenting scene-wide labeling information. Subsequently, we introduce SAM features via a transformer into the INF of the scene, supplementing the semantic information. The experimental results demonstrate that our method outperforms the mainstream method, confirming the efficacy of SAM as a supplement to the INF for this task. |
| title | Multi-view Remote Sensing Image Segmentation With SAM priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.14171 |