VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909619173982208 |
|---|---|
| author | Zhang, Haiming Zhou, Wending Zhu, Yiyao Yan, Xu Gao, Jiantao Bai, Dongfeng Cai, Yingjie Liu, Bingbing Cui, Shuguang Li, Zhen |
| author_facet | Zhang, Haiming Zhou, Wending Zhu, Yiyao Yan, Xu Gao, Jiantao Bai, Dongfeng Cai, Yingjie Liu, Bingbing Cui, Shuguang Li, Zhen |
| contents | This paper introduces VisionPAD, a novel self-supervised pre-training paradigm designed for vision-centric algorithms in autonomous driving. In contrast to previous approaches that employ neural rendering with explicit depth supervision, VisionPAD utilizes more efficient 3D Gaussian Splatting to reconstruct multi-view representations using only images as supervision. Specifically, we introduce a self-supervised method for voxel velocity estimation. By warping voxels to adjacent frames and supervising the rendered outputs, the model effectively learns motion cues in the sequential data. Furthermore, we adopt a multi-frame photometric consistency approach to enhance geometric perception. It projects adjacent frames to the current frame based on rendered depths and relative poses, boosting the 3D geometric representation through pure image supervision. Extensive experiments on autonomous driving datasets demonstrate that VisionPAD significantly improves performance in 3D object detection, occupancy prediction and map segmentation, surpassing state-of-the-art pre-training strategies by a considerable margin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_14716 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving Zhang, Haiming Zhou, Wending Zhu, Yiyao Yan, Xu Gao, Jiantao Bai, Dongfeng Cai, Yingjie Liu, Bingbing Cui, Shuguang Li, Zhen Computer Vision and Pattern Recognition Machine Learning Robotics This paper introduces VisionPAD, a novel self-supervised pre-training paradigm designed for vision-centric algorithms in autonomous driving. In contrast to previous approaches that employ neural rendering with explicit depth supervision, VisionPAD utilizes more efficient 3D Gaussian Splatting to reconstruct multi-view representations using only images as supervision. Specifically, we introduce a self-supervised method for voxel velocity estimation. By warping voxels to adjacent frames and supervising the rendered outputs, the model effectively learns motion cues in the sequential data. Furthermore, we adopt a multi-frame photometric consistency approach to enhance geometric perception. It projects adjacent frames to the current frame based on rendered depths and relative poses, boosting the 3D geometric representation through pure image supervision. Extensive experiments on autonomous driving datasets demonstrate that VisionPAD significantly improves performance in 3D object detection, occupancy prediction and map segmentation, surpassing state-of-the-art pre-training strategies by a considerable margin. |
| title | VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving |
| topic | Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2411.14716 |