MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908637011640320 |
|---|---|
| author | Gu, Qiwen Wang, Xufei Zhao, Junqiao Tao, Siyue Feng, Tiantian Wang, Ziqiao Chen, Guang |
| author_facet | Gu, Qiwen Wang, Xufei Zhao, Junqiao Tao, Siyue Feng, Tiantian Wang, Ziqiao Chen, Guang |
| contents | Visual Place Recognition (VPR) enables robust localization through image retrieval based on learned descriptors.
However, drastic appearance variations of images at the same place caused by viewpoint changes can lead to inconsistent supervision signals, thereby degrading descriptor learning.
Existing methods either rely on manually defined cropping rules or labeled data for view differentiation, but they suffer from two major limitations:
(1) reliance on labels or handcrafted rules restricts generalization capability;
(2) even within the same view direction, occlusions can introduce feature ambiguity.
To address these issues, we propose MutualVPR, a mutual learning framework that integrates unsupervised view self-classification and descriptor learning.
We first group images by geographic coordinates, then iteratively refine the clusters using K-means to dynamically assign place categories without orientation labels.
Specifically, we adopt a DINOv2-based encoder to initialize the clustering.
During training, the encoder and clustering co-evolve, progressively separating drastic appearance variations of the same place and enabling consistent supervision.
Furthermore, we find that capturing fine-grained image differences at a place enhances robustness.
Experiments demonstrate that MutualVPR achieves state-of-the-art (SOTA) performance across multiple datasets, validating the effectiveness of our framework in improving view direction generalization, occlusion robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_09199 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering Gu, Qiwen Wang, Xufei Zhao, Junqiao Tao, Siyue Feng, Tiantian Wang, Ziqiao Chen, Guang Computer Vision and Pattern Recognition Visual Place Recognition (VPR) enables robust localization through image retrieval based on learned descriptors. However, drastic appearance variations of images at the same place caused by viewpoint changes can lead to inconsistent supervision signals, thereby degrading descriptor learning. Existing methods either rely on manually defined cropping rules or labeled data for view differentiation, but they suffer from two major limitations: (1) reliance on labels or handcrafted rules restricts generalization capability; (2) even within the same view direction, occlusions can introduce feature ambiguity. To address these issues, we propose MutualVPR, a mutual learning framework that integrates unsupervised view self-classification and descriptor learning. We first group images by geographic coordinates, then iteratively refine the clusters using K-means to dynamically assign place categories without orientation labels. Specifically, we adopt a DINOv2-based encoder to initialize the clustering. During training, the encoder and clustering co-evolve, progressively separating drastic appearance variations of the same place and enabling consistent supervision. Furthermore, we find that capturing fine-grained image differences at a place enhances robustness. Experiments demonstrate that MutualVPR achieves state-of-the-art (SOTA) performance across multiple datasets, validating the effectiveness of our framework in improving view direction generalization, occlusion robustness. |
| title | MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.09199 |