Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.09971 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909695130730496 |
|---|---|
| author | Lin, Mengying Liu, Shugao Zhang, Dingxi Chen, Yaran Wang, Zhaoran Li, Haoran Zhao, Dongbin |
| author_facet | Lin, Mengying Liu, Shugao Zhang, Dingxi Chen, Yaran Wang, Zhaoran Li, Haoran Zhao, Dongbin |
| contents | Object-goal navigation requires mobile robots to efficiently locate targets with visual and spatial information, yet existing methods struggle with generalization in unseen environments. Heuristic approaches with naive metrics fail in complex layouts, while graph-based and learning-based methods suffer from environmental biases and limited generalization. Although Large Language Models (LLMs) as planners or agents offer a rich knowledge base, they are cost-inefficient and lack targeted historical experience. To address these challenges, we propose the LLM-enhanced Object Affinities Transfer (LOAT) framework, integrating LLM-derived semantics with learning-based approaches to leverage experiential object affinities for better generalization in unseen settings. LOAT employs a dual-module strategy: one module accesses LLMs' vast knowledge, and the other applies learned object semantic relationships, dynamically fusing these sources based on context. Evaluations in AI2-THOR and Habitat simulators show significant improvements in navigation success and efficiency, and real-world deployment demonstrates the zero-shot ability of LOAT to enhance object-goal navigation systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_09971 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Advancing Object Goal Navigation Through LLM-enhanced Object Affinities Transfer Lin, Mengying Liu, Shugao Zhang, Dingxi Chen, Yaran Wang, Zhaoran Li, Haoran Zhao, Dongbin Robotics Object-goal navigation requires mobile robots to efficiently locate targets with visual and spatial information, yet existing methods struggle with generalization in unseen environments. Heuristic approaches with naive metrics fail in complex layouts, while graph-based and learning-based methods suffer from environmental biases and limited generalization. Although Large Language Models (LLMs) as planners or agents offer a rich knowledge base, they are cost-inefficient and lack targeted historical experience. To address these challenges, we propose the LLM-enhanced Object Affinities Transfer (LOAT) framework, integrating LLM-derived semantics with learning-based approaches to leverage experiential object affinities for better generalization in unseen settings. LOAT employs a dual-module strategy: one module accesses LLMs' vast knowledge, and the other applies learned object semantic relationships, dynamically fusing these sources based on context. Evaluations in AI2-THOR and Habitat simulators show significant improvements in navigation success and efficiency, and real-world deployment demonstrates the zero-shot ability of LOAT to enhance object-goal navigation systems. |
| title | Advancing Object Goal Navigation Through LLM-enhanced Object Affinities Transfer |
| topic | Robotics |
| url | https://arxiv.org/abs/2403.09971 |