Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.15977 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910387699449856 |
|---|---|
| author | Xia, Yan Shi, Letian Ding, Zifeng Henriques, João F. Cremers, Daniel |
| author_facet | Xia, Yan Shi, Letian Ding, Zifeng Henriques, João F. Cremers, Daniel |
| contents | We tackle the problem of 3D point cloud localization based on a few natural linguistic descriptions and introduce a novel neural network, Text2Loc, that fully interprets the semantic relationship between points and text. Text2Loc follows a coarse-to-fine localization pipeline: text-submap global place recognition, followed by fine localization. In global place recognition, relational dynamics among each textual hint are captured in a hierarchical transformer with max-pooling (HTM), whereas a balance between positive and negative pairs is maintained using text-submap contrastive learning. Moreover, we propose a novel matching-free fine localization method to further refine the location predictions, which completely removes the need for complicated text-instance matching and is lighter, faster, and more accurate than previous methods. Extensive experiments show that Text2Loc improves the localization accuracy by up to $2\times$ over the state-of-the-art on the KITTI360Pose dataset. Our project page is publicly available at \url{https://yan-xia.github.io/projects/text2loc/}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_15977 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Text2Loc: 3D Point Cloud Localization from Natural Language Xia, Yan Shi, Letian Ding, Zifeng Henriques, João F. Cremers, Daniel Computer Vision and Pattern Recognition We tackle the problem of 3D point cloud localization based on a few natural linguistic descriptions and introduce a novel neural network, Text2Loc, that fully interprets the semantic relationship between points and text. Text2Loc follows a coarse-to-fine localization pipeline: text-submap global place recognition, followed by fine localization. In global place recognition, relational dynamics among each textual hint are captured in a hierarchical transformer with max-pooling (HTM), whereas a balance between positive and negative pairs is maintained using text-submap contrastive learning. Moreover, we propose a novel matching-free fine localization method to further refine the location predictions, which completely removes the need for complicated text-instance matching and is lighter, faster, and more accurate than previous methods. Extensive experiments show that Text2Loc improves the localization accuracy by up to $2\times$ over the state-of-the-art on the KITTI360Pose dataset. Our project page is publicly available at \url{https://yan-xia.github.io/projects/text2loc/}. |
| title | Text2Loc: 3D Point Cloud Localization from Natural Language |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2311.15977 |