Spatial Retrieval Augmented Autonomous Driving
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914185616556032 |
|---|---|
| author | Jia, Xiaosong Zhang, Chenhe Jiang, Yule Wong, Songbur Zhang, Zhiyuan Chen, Chen Zhang, Shaofeng Zhou, Xuanhe Yang, Xue Yan, Junchi Jiang, Yu-Gang |
| author_facet | Jia, Xiaosong Zhang, Chenhe Jiang, Yule Wong, Songbur Zhang, Zhiyuan Chen, Chen Zhang, Shaofeng Zhou, Xuanhe Yang, Xue Yan, Junchi Jiang, Yu-Gang |
| contents | Existing autonomous driving systems rely on onboard sensors (cameras, LiDAR, IMU, etc) for environmental perception. However, this paradigm is limited by the drive-time perception horizon and often fails under limited view scope, occlusion or extreme conditions such as darkness and rain. In contrast, human drivers are able to recall road structure even under poor visibility. To endow models with this ``recall" ability, we propose the spatial retrieval paradigm, introducing offline retrieved geographic images as an additional input. These images are easy to obtain from offline caches (e.g, Google Maps or stored autonomous driving datasets) without requiring additional sensors, making it a plug-and-play extension for existing AD tasks.
For experiments, we first extend the nuScenes dataset with geographic images retrieved via Google Maps APIs and align the new data with ego-vehicle trajectories. We establish baselines across five core autonomous driving tasks: object detection, online mapping, occupancy prediction, end-to-end planning, and generative world modeling. Extensive experiments show that the extended modality could enhance the performance of certain tasks. We will open-source dataset curation code, data, and benchmarks for further study of this new autonomous driving paradigm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_06865 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Spatial Retrieval Augmented Autonomous Driving Jia, Xiaosong Zhang, Chenhe Jiang, Yule Wong, Songbur Zhang, Zhiyuan Chen, Chen Zhang, Shaofeng Zhou, Xuanhe Yang, Xue Yan, Junchi Jiang, Yu-Gang Computer Vision and Pattern Recognition Existing autonomous driving systems rely on onboard sensors (cameras, LiDAR, IMU, etc) for environmental perception. However, this paradigm is limited by the drive-time perception horizon and often fails under limited view scope, occlusion or extreme conditions such as darkness and rain. In contrast, human drivers are able to recall road structure even under poor visibility. To endow models with this ``recall" ability, we propose the spatial retrieval paradigm, introducing offline retrieved geographic images as an additional input. These images are easy to obtain from offline caches (e.g, Google Maps or stored autonomous driving datasets) without requiring additional sensors, making it a plug-and-play extension for existing AD tasks. For experiments, we first extend the nuScenes dataset with geographic images retrieved via Google Maps APIs and align the new data with ego-vehicle trajectories. We establish baselines across five core autonomous driving tasks: object detection, online mapping, occupancy prediction, end-to-end planning, and generative world modeling. Extensive experiments show that the extended modality could enhance the performance of certain tasks. We will open-source dataset curation code, data, and benchmarks for further study of this new autonomous driving paradigm. |
| title | Spatial Retrieval Augmented Autonomous Driving |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.06865 |