Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918150780485632 |
|---|---|
| author | Zhang, Yue Ma, Tianyi Wang, Zun Qiao, Yanyuan Kordjamshidi, Parisa |
| author_facet | Zhang, Yue Ma, Tianyi Wang, Zun Qiao, Yanyuan Kordjamshidi, Parisa |
| contents | Integrating large language models (LLMs) into embodied AI models is becoming increasingly prevalent. However, existing zero-shot LLM-based Vision-and-Language Navigation (VLN) agents either encode images as textual scene descriptions, potentially oversimplifying visual details, or process raw image inputs, which can fail to capture abstract semantics required for high-level reasoning. In this paper, we improve the navigation agent's contextual understanding by incorporating textual descriptions from multiple perspectives that facilitate analogical reasoning across images. By leveraging text-based analogical reasoning, the agent enhances its global scene understanding and spatial reasoning, leading to more accurate action decisions. We evaluate our approach on the R2R dataset, where our experiments demonstrate significant improvements in navigation performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25139 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs Zhang, Yue Ma, Tianyi Wang, Zun Qiao, Yanyuan Kordjamshidi, Parisa Artificial Intelligence Computer Vision and Pattern Recognition Multimedia Integrating large language models (LLMs) into embodied AI models is becoming increasingly prevalent. However, existing zero-shot LLM-based Vision-and-Language Navigation (VLN) agents either encode images as textual scene descriptions, potentially oversimplifying visual details, or process raw image inputs, which can fail to capture abstract semantics required for high-level reasoning. In this paper, we improve the navigation agent's contextual understanding by incorporating textual descriptions from multiple perspectives that facilitate analogical reasoning across images. By leveraging text-based analogical reasoning, the agent enhances its global scene understanding and spatial reasoning, leading to more accurate action decisions. We evaluate our approach on the R2R dataset, where our experiments demonstrate significant improvements in navigation performance. |
| title | Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2509.25139 |