Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914679077470208 |
|---|---|
| author | Kalantidis, Yannis Sarıyıldız, Mert Bülent Rezende, Rafael S. Weinzaepfel, Philippe Larlus, Diane Csurka, Gabriela |
| author_facet | Kalantidis, Yannis Sarıyıldız, Mert Bülent Rezende, Rafael S. Weinzaepfel, Philippe Larlus, Diane Csurka, Gabriela |
| contents | State-of-the-art visual localization approaches generally rely on a first image retrieval step whose role is crucial. Yet, retrieval often struggles when facing varying conditions, due to e.g. weather or time of day, with dramatic consequences on the visual localization accuracy. In this paper, we improve this retrieval step and tailor it to the final localization task. Among the several changes we advocate for, we propose to synthesize variants of the training set images, obtained from generative text-to-image models, in order to automatically expand the training set towards a number of nameable variations that particularly hurt visual localization. After expanding the training set, we propose a training approach that leverages the specificities and the underlying geometry of this mix of real and synthetic images. We experimentally show that those changes translate into large improvements for the most challenging visual localization datasets. Project page: https://europe.naverlabs.com/ret4loc |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_09237 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency Kalantidis, Yannis Sarıyıldız, Mert Bülent Rezende, Rafael S. Weinzaepfel, Philippe Larlus, Diane Csurka, Gabriela Computer Vision and Pattern Recognition State-of-the-art visual localization approaches generally rely on a first image retrieval step whose role is crucial. Yet, retrieval often struggles when facing varying conditions, due to e.g. weather or time of day, with dramatic consequences on the visual localization accuracy. In this paper, we improve this retrieval step and tailor it to the final localization task. Among the several changes we advocate for, we propose to synthesize variants of the training set images, obtained from generative text-to-image models, in order to automatically expand the training set towards a number of nameable variations that particularly hurt visual localization. After expanding the training set, we propose a training approach that leverages the specificities and the underlying geometry of this mix of real and synthetic images. We experimentally show that those changes translate into large improvements for the most challenging visual localization datasets. Project page: https://europe.naverlabs.com/ret4loc |
| title | Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.09237 |