Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.12079 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910747812954112 |
|---|---|
| author | Xia, Yan Li, Zhendong Li, Yun-Jin Shi, Letian Cao, Hu Henriques, João F. Cremers, Daniel |
| author_facet | Xia, Yan Li, Zhendong Li, Yun-Jin Shi, Letian Cao, Hu Henriques, João F. Cremers, Daniel |
| contents | To date, most place recognition methods focus on single-modality retrieval. While they perform well in specific environments, cross-modal methods offer greater flexibility by allowing seamless switching between map and query sources. It also promises to reduce computation requirements by having a unified model, and achieving greater sample efficiency by sharing parameters. In this work, we develop a universal solution to place recognition, UniLoc, that works with any single query modality (natural language, image, or point cloud). UniLoc leverages recent advances in large-scale contrastive learning, and learns by matching hierarchically at two levels: instance-level matching and scene-level matching. Specifically, we propose a novel Self-Attention based Pooling (SAP) module to evaluate the importance of instance descriptors when aggregated into a place-level descriptor. Experiments on the KITTI-360 dataset demonstrate the benefits of cross-modality for place recognition, achieving superior performance in cross-modal settings and competitive results also for uni-modal scenarios. Our project page is publicly available at https://yan-xia.github.io/projects/UniLoc/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12079 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | UniLoc: Towards Universal Place Recognition Using Any Single Modality Xia, Yan Li, Zhendong Li, Yun-Jin Shi, Letian Cao, Hu Henriques, João F. Cremers, Daniel Computer Vision and Pattern Recognition To date, most place recognition methods focus on single-modality retrieval. While they perform well in specific environments, cross-modal methods offer greater flexibility by allowing seamless switching between map and query sources. It also promises to reduce computation requirements by having a unified model, and achieving greater sample efficiency by sharing parameters. In this work, we develop a universal solution to place recognition, UniLoc, that works with any single query modality (natural language, image, or point cloud). UniLoc leverages recent advances in large-scale contrastive learning, and learns by matching hierarchically at two levels: instance-level matching and scene-level matching. Specifically, we propose a novel Self-Attention based Pooling (SAP) module to evaluate the importance of instance descriptors when aggregated into a place-level descriptor. Experiments on the KITTI-360 dataset demonstrate the benefits of cross-modality for place recognition, achieving superior performance in cross-modal settings and competitive results also for uni-modal scenarios. Our project page is publicly available at https://yan-xia.github.io/projects/UniLoc/. |
| title | UniLoc: Towards Universal Place Recognition Using Any Single Modality |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.12079 |