Robot Navigation Using Physically Grounded Vision-Language Models in Outdoor Environments
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929521910874112 |
|---|---|
| author | Elnoor, Mohamed Weerakoon, Kasun Seneviratne, Gershom Xian, Ruiqi Guan, Tianrui Jaffar, Mohamed Khalid M Rajagopal, Vignesh Manocha, Dinesh |
| author_facet | Elnoor, Mohamed Weerakoon, Kasun Seneviratne, Gershom Xian, Ruiqi Guan, Tianrui Jaffar, Mohamed Khalid M Rajagopal, Vignesh Manocha, Dinesh |
| contents | We present a novel autonomous robot navigation algorithm for outdoor environments that is capable of handling diverse terrain traversability conditions. Our approach, VLM-GroNav, uses vision-language models (VLMs) and integrates them with physical grounding that is used to assess intrinsic terrain properties such as deformability and slipperiness. We use proprioceptive-based sensing, which provides direct measurements of these physical properties, and enhances the overall semantic understanding of the terrains. Our formulation uses in-context learning to ground the VLM's semantic understanding with proprioceptive data to allow dynamic updates of traversability estimates based on the robot's real-time physical interactions with the environment. We use the updated traversability estimations to inform both the local and global planners for real-time trajectory replanning. We validate our method on a legged robot (Ghost Vision 60) and a wheeled robot (Clearpath Husky), in diverse real-world outdoor environments with different deformable and slippery terrains. In practice, we observe significant improvements over state-of-the-art methods by up to 50% increase in navigation success rate. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_20445 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robot Navigation Using Physically Grounded Vision-Language Models in Outdoor Environments Elnoor, Mohamed Weerakoon, Kasun Seneviratne, Gershom Xian, Ruiqi Guan, Tianrui Jaffar, Mohamed Khalid M Rajagopal, Vignesh Manocha, Dinesh Robotics We present a novel autonomous robot navigation algorithm for outdoor environments that is capable of handling diverse terrain traversability conditions. Our approach, VLM-GroNav, uses vision-language models (VLMs) and integrates them with physical grounding that is used to assess intrinsic terrain properties such as deformability and slipperiness. We use proprioceptive-based sensing, which provides direct measurements of these physical properties, and enhances the overall semantic understanding of the terrains. Our formulation uses in-context learning to ground the VLM's semantic understanding with proprioceptive data to allow dynamic updates of traversability estimates based on the robot's real-time physical interactions with the environment. We use the updated traversability estimations to inform both the local and global planners for real-time trajectory replanning. We validate our method on a legged robot (Ghost Vision 60) and a wheeled robot (Clearpath Husky), in diverse real-world outdoor environments with different deformable and slippery terrains. In practice, we observe significant improvements over state-of-the-art methods by up to 50% increase in navigation success rate. |
| title | Robot Navigation Using Physically Grounded Vision-Language Models in Outdoor Environments |
| topic | Robotics |
| url | https://arxiv.org/abs/2409.20445 |