Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.19972 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918145982201856 |
|---|---|
| author | Yang, Zhen Dong, Yanpeng Wang, Jiayu Wang, Heng Ma, Lichao Cui, Zijian Liu, Qi Pei, Haoran Zhang, Kexin Zhang, Chao |
| author_facet | Yang, Zhen Dong, Yanpeng Wang, Jiayu Wang, Heng Ma, Lichao Cui, Zijian Liu, Qi Pei, Haoran Zhang, Kexin Zhang, Chao |
| contents | Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19972 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction Yang, Zhen Dong, Yanpeng Wang, Jiayu Wang, Heng Ma, Lichao Cui, Zijian Liu, Qi Pei, Haoran Zhang, Kexin Zhang, Chao Computer Vision and Pattern Recognition Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc. |
| title | DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.19972 |