Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2410.03900 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909337006374912 |
|---|---|
| author | Pate, Seth Wong, Lawson L. S. |
| author_facet | Pate, Seth Wong, Lawson L. S. |
| contents | We study the task of locating a user in a mapped indoor environment using natural language queries and images from the environment.
Building on recent pretrained vision-language models, we learn a similarity score between text descriptions and images of locations in the environment.
This score allows us to identify locations that best match the language query, estimating the user's location.
Our approach is capable of localizing on environments, text, and images that were not seen during training.
One model, finetuned CLIP, outperformed humans in our evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03900 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Wallpaper is Ugly: Indoor Localization using Vision and Language Pate, Seth Wong, Lawson L. S. Computer Vision and Pattern Recognition We study the task of locating a user in a mapped indoor environment using natural language queries and images from the environment. Building on recent pretrained vision-language models, we learn a similarity score between text descriptions and images of locations in the environment. This score allows us to identify locations that best match the language query, estimating the user's location. Our approach is capable of localizing on environments, text, and images that were not seen during training. One model, finetuned CLIP, outperformed humans in our evaluation. |
| title | The Wallpaper is Ugly: Indoor Localization using Vision and Language |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.03900 |