CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908330736222208 |
|---|---|
| author | Liu, Xinhao Li, Jintong Jiang, Yicheng Sujay, Niranjan Yang, Zhicheng Zhang, Juexiao Abanes, John Zhang, Jing Feng, Chen |
| author_facet | Liu, Xinhao Li, Jintong Jiang, Yicheng Sujay, Niranjan Yang, Zhicheng Zhang, Juexiao Abanes, John Zhang, Jing Feng, Chen |
| contents | Navigating dynamic urban environments presents significant challenges for embodied agents, requiring advanced spatial reasoning and adherence to common-sense norms. Despite progress, existing visual navigation methods struggle in map-free or off-street settings, limiting the deployment of autonomous agents like last-mile delivery robots. To overcome these obstacles, we propose a scalable, data-driven approach for human-like urban navigation by training agents on thousands of hours of in-the-wild city walking and driving videos sourced from the web. We introduce a simple and scalable data processing pipeline that extracts action supervision from these videos, enabling large-scale imitation learning without costly annotations. Our model learns sophisticated navigation policies to handle diverse challenges and critical scenarios. Experimental results show that training on large-scale, diverse datasets significantly enhances navigation performance, surpassing current methods. This work shows the potential of using abundant online video data to develop robust navigation policies for embodied agents in dynamic urban settings. Project homepage is at https://ai4ce.github.io/CityWalker/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17820 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos Liu, Xinhao Li, Jintong Jiang, Yicheng Sujay, Niranjan Yang, Zhicheng Zhang, Juexiao Abanes, John Zhang, Jing Feng, Chen Computer Vision and Pattern Recognition Robotics Navigating dynamic urban environments presents significant challenges for embodied agents, requiring advanced spatial reasoning and adherence to common-sense norms. Despite progress, existing visual navigation methods struggle in map-free or off-street settings, limiting the deployment of autonomous agents like last-mile delivery robots. To overcome these obstacles, we propose a scalable, data-driven approach for human-like urban navigation by training agents on thousands of hours of in-the-wild city walking and driving videos sourced from the web. We introduce a simple and scalable data processing pipeline that extracts action supervision from these videos, enabling large-scale imitation learning without costly annotations. Our model learns sophisticated navigation policies to handle diverse challenges and critical scenarios. Experimental results show that training on large-scale, diverse datasets significantly enhances navigation performance, surpassing current methods. This work shows the potential of using abundant online video data to develop robust navigation policies for embodied agents in dynamic urban settings. Project homepage is at https://ai4ce.github.io/CityWalker/. |
| title | CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2411.17820 |