SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912134397427712 |
|---|---|
| author | Dong, Hao Gu, Weihao Zhang, Xianjing Xu, Jintao Ai, Rui Lu, Huimin Kannala, Juho Chen, Xieyuanli |
| author_facet | Dong, Hao Gu, Weihao Zhang, Xianjing Xu, Jintao Ai, Rui Lu, Huimin Kannala, Juho Chen, Xieyuanli |
| contents | High-definition (HD) semantic map generation of the environment is an essential component of autonomous driving. Existing methods have achieved good performance in this task by fusing different sensor modalities, such as LiDAR and camera. However, current works are based on raw data or network feature-level fusion and only consider short-range HD map generation, limiting their deployment to realistic autonomous driving applications. In this paper, we focus on the task of building the HD maps in both short ranges, i.e., within 30 m, and also predicting long-range HD maps up to 90 m, which is required by downstream path planning and control tasks to improve the smoothness and safety of autonomous driving. To this end, we propose a novel network named SuperFusion, exploiting the fusion of LiDAR and camera data at multiple levels. We use LiDAR depth to improve image depth estimation and use image features to guide long-range LiDAR feature prediction. We benchmark our SuperFusion on the nuScenes dataset and a self-recorded dataset and show that it outperforms the state-of-the-art baseline methods with large margins on all intervals. Additionally, we apply the generated HD map to a downstream path planning task, demonstrating that the long-range HD maps predicted by our method can lead to better path planning for autonomous vehicles. Our code has been released at https://github.com/haomo-ai/SuperFusion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_15656 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation Dong, Hao Gu, Weihao Zhang, Xianjing Xu, Jintao Ai, Rui Lu, Huimin Kannala, Juho Chen, Xieyuanli Computer Vision and Pattern Recognition Robotics High-definition (HD) semantic map generation of the environment is an essential component of autonomous driving. Existing methods have achieved good performance in this task by fusing different sensor modalities, such as LiDAR and camera. However, current works are based on raw data or network feature-level fusion and only consider short-range HD map generation, limiting their deployment to realistic autonomous driving applications. In this paper, we focus on the task of building the HD maps in both short ranges, i.e., within 30 m, and also predicting long-range HD maps up to 90 m, which is required by downstream path planning and control tasks to improve the smoothness and safety of autonomous driving. To this end, we propose a novel network named SuperFusion, exploiting the fusion of LiDAR and camera data at multiple levels. We use LiDAR depth to improve image depth estimation and use image features to guide long-range LiDAR feature prediction. We benchmark our SuperFusion on the nuScenes dataset and a self-recorded dataset and show that it outperforms the state-of-the-art baseline methods with large margins on all intervals. Additionally, we apply the generated HD map to a downstream path planning task, demonstrating that the long-range HD maps predicted by our method can lead to better path planning for autonomous vehicles. Our code has been released at https://github.com/haomo-ai/SuperFusion. |
| title | SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2211.15656 |