Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916189469409280 |
|---|---|
| author | Wang, Zihan Li, Xiangyang Yang, Jiahao Liu, Yeqi Hu, Junjie Jiang, Ming Jiang, Shuqiang |
| author_facet | Wang, Zihan Li, Xiangyang Yang, Jiahao Liu, Yeqi Hu, Junjie Jiang, Ming Jiang, Shuqiang |
| contents | Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. At each navigation step, the agent selects from possible candidate locations and then makes the move. For better navigation planning, the lookahead exploration strategy aims to effectively evaluate the agent's next action by accurately anticipating the future environment of candidate locations. To this end, some existing works predict RGB images for future environments, while this strategy suffers from image distortion and high computational cost. To address these issues, we propose the pre-trained hierarchical neural radiance representation model (HNR) to produce multi-level semantic features for future environments, which are more robust and efficient than pixel-wise RGB reconstruction. Furthermore, with the predicted future environmental representations, our lookahead VLN model is able to construct the navigable future path tree and select the optimal path via efficient parallel evaluation. Extensive experiments on the VLN-CE datasets confirm the effectiveness of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01943 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation Wang, Zihan Li, Xiangyang Yang, Jiahao Liu, Yeqi Hu, Junjie Jiang, Ming Jiang, Shuqiang Computer Vision and Pattern Recognition Robotics Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. At each navigation step, the agent selects from possible candidate locations and then makes the move. For better navigation planning, the lookahead exploration strategy aims to effectively evaluate the agent's next action by accurately anticipating the future environment of candidate locations. To this end, some existing works predict RGB images for future environments, while this strategy suffers from image distortion and high computational cost. To address these issues, we propose the pre-trained hierarchical neural radiance representation model (HNR) to produce multi-level semantic features for future environments, which are more robust and efficient than pixel-wise RGB reconstruction. Furthermore, with the predicted future environmental representations, our lookahead VLN model is able to construct the navigable future path tree and select the optimal path via efficient parallel evaluation. Extensive experiments on the VLN-CE datasets confirm the effectiveness of our method. |
| title | Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2404.01943 |