Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.14507 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909651537231872 |
|---|---|
| author | Subedi, Nitesh Haroon, Adam Ganguly, Shreyan Tetteh, Samuel T. K. Koirala, Prajwal Fleming, Cody Sarkar, Soumik |
| author_facet | Subedi, Nitesh Haroon, Adam Ganguly, Shreyan Tetteh, Samuel T. K. Koirala, Prajwal Fleming, Cody Sarkar, Soumik |
| contents | Foundation models have revolutionized robotics by providing rich semantic representations without task-specific training. While many approaches integrate pretrained vision-language models (VLMs) with specialized navigation architectures, the fundamental question remains: can these pretrained embeddings alone successfully guide navigation without additional fine-tuning or specialized modules? We present a minimalist framework that decouples this question by training a behavior cloning policy directly on frozen vision-language embeddings from demonstrations collected by a privileged expert. Our approach achieves a 74% success rate in navigation to language-specified targets, compared to 100% for the state-aware expert, though requiring 3.2 times more steps on average. This performance gap reveals that pretrained embeddings effectively support basic language grounding but struggle with long-horizon planning and spatial reasoning. By providing this empirical baseline, we highlight both the capabilities and limitations of using foundation models as drop-in representations for embodied tasks, offering critical insights for robotics researchers facing practical design tradeoffs between system complexity and performance in resource-constrained scenarios. Our code is available at https://github.com/oadamharoon/text2nav |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14507 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can Pretrained Vision-Language Embeddings Alone Guide Robot Navigation? Subedi, Nitesh Haroon, Adam Ganguly, Shreyan Tetteh, Samuel T. K. Koirala, Prajwal Fleming, Cody Sarkar, Soumik Robotics Foundation models have revolutionized robotics by providing rich semantic representations without task-specific training. While many approaches integrate pretrained vision-language models (VLMs) with specialized navigation architectures, the fundamental question remains: can these pretrained embeddings alone successfully guide navigation without additional fine-tuning or specialized modules? We present a minimalist framework that decouples this question by training a behavior cloning policy directly on frozen vision-language embeddings from demonstrations collected by a privileged expert. Our approach achieves a 74% success rate in navigation to language-specified targets, compared to 100% for the state-aware expert, though requiring 3.2 times more steps on average. This performance gap reveals that pretrained embeddings effectively support basic language grounding but struggle with long-horizon planning and spatial reasoning. By providing this empirical baseline, we highlight both the capabilities and limitations of using foundation models as drop-in representations for embodied tasks, offering critical insights for robotics researchers facing practical design tradeoffs between system complexity and performance in resource-constrained scenarios. Our code is available at https://github.com/oadamharoon/text2nav |
| title | Can Pretrained Vision-Language Embeddings Alone Guide Robot Navigation? |
| topic | Robotics |
| url | https://arxiv.org/abs/2506.14507 |