TP3M: Transformer-based Pseudo 3D Image Matching with Reference Image
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917745807851520 |
|---|---|
| author | Han, Liming Liu, Zhaoxiang Lian, Shiguo |
| author_facet | Han, Liming Liu, Zhaoxiang Lian, Shiguo |
| contents | Image matching is still challenging in such scenes with large viewpoints or illumination changes or with low textures. In this paper, we propose a Transformer-based pseudo 3D image matching method. It upgrades the 2D features extracted from the source image to 3D features with the help of a reference image and matches to the 2D features extracted from the destination image by the coarse-to-fine 3D matching. Our key discovery is that by introducing the reference image, the source image's fine points are screened and furtherly their feature descriptors are enriched from 2D to 3D, which improves the match performance with the destination image. Experimental results on multiple datasets show that the proposed method achieves the state-of-the-art on the tasks of homography estimation, pose estimation and visual localization especially in challenging scenes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_08434 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TP3M: Transformer-based Pseudo 3D Image Matching with Reference Image Han, Liming Liu, Zhaoxiang Lian, Shiguo Computer Vision and Pattern Recognition Image matching is still challenging in such scenes with large viewpoints or illumination changes or with low textures. In this paper, we propose a Transformer-based pseudo 3D image matching method. It upgrades the 2D features extracted from the source image to 3D features with the help of a reference image and matches to the 2D features extracted from the destination image by the coarse-to-fine 3D matching. Our key discovery is that by introducing the reference image, the source image's fine points are screened and furtherly their feature descriptors are enriched from 2D to 3D, which improves the match performance with the destination image. Experimental results on multiple datasets show that the proposed method achieves the state-of-the-art on the tasks of homography estimation, pose estimation and visual localization especially in challenging scenes. |
| title | TP3M: Transformer-based Pseudo 3D Image Matching with Reference Image |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.08434 |