A Light-weight Transformer-based Self-supervised Matching Network for Heterogeneous Images
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929331650953216 |
|---|---|
| author | Zhang, Wang Li, Tingting Zhang, Yuntian Pei, Gensheng Jiang, Xiruo Yao, Yazhou |
| author_facet | Zhang, Wang Li, Tingting Zhang, Yuntian Pei, Gensheng Jiang, Xiruo Yao, Yazhou |
| contents | Matching visible and near-infrared (NIR) images remains a significant challenge in remote sensing image fusion. The nonlinear radiometric differences between heterogeneous remote sensing images make the image matching task even more difficult. Deep learning has gained substantial attention in computer vision tasks in recent years. However, many methods rely on supervised learning and necessitate large amounts of annotated data. Nevertheless, annotated data is frequently limited in the field of remote sensing image matching. To address this challenge, this paper proposes a novel keypoint descriptor approach that obtains robust feature descriptors via a self-supervised matching network. A light-weight transformer network, termed as LTFormer, is designed to generate deep-level feature descriptors. Furthermore, we implement an innovative triplet loss function, LT Loss, to enhance the matching performance further. Our approach outperforms conventional hand-crafted local feature descriptors and proves equally competitive compared to state-of-the-art deep learning-based methods, even amidst the shortage of annotated data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_19311 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Light-weight Transformer-based Self-supervised Matching Network for Heterogeneous Images Zhang, Wang Li, Tingting Zhang, Yuntian Pei, Gensheng Jiang, Xiruo Yao, Yazhou Computer Vision and Pattern Recognition Multimedia Matching visible and near-infrared (NIR) images remains a significant challenge in remote sensing image fusion. The nonlinear radiometric differences between heterogeneous remote sensing images make the image matching task even more difficult. Deep learning has gained substantial attention in computer vision tasks in recent years. However, many methods rely on supervised learning and necessitate large amounts of annotated data. Nevertheless, annotated data is frequently limited in the field of remote sensing image matching. To address this challenge, this paper proposes a novel keypoint descriptor approach that obtains robust feature descriptors via a self-supervised matching network. A light-weight transformer network, termed as LTFormer, is designed to generate deep-level feature descriptors. Furthermore, we implement an innovative triplet loss function, LT Loss, to enhance the matching performance further. Our approach outperforms conventional hand-crafted local feature descriptors and proves equally competitive compared to state-of-the-art deep learning-based methods, even amidst the shortage of annotated data. |
| title | A Light-weight Transformer-based Self-supervised Matching Network for Heterogeneous Images |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2404.19311 |