MonoOcc: Digging into Monocular Semantic Occupancy Prediction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913264160473088 |
|---|---|
| author | Zheng, Yupeng Li, Xiang Li, Pengfei Zheng, Yuhang Jin, Bu Zhong, Chengliang Long, Xiaoxiao Zhao, Hao Zhang, Qichao |
| author_facet | Zheng, Yupeng Li, Xiang Li, Pengfei Zheng, Yuhang Jin, Bu Zhong, Chengliang Long, Xiaoxiao Zhao, Hao Zhang, Qichao |
| contents | Monocular Semantic Occupancy Prediction aims to infer the complete 3D geometry and semantic information of scenes from only 2D images. It has garnered significant attention, particularly due to its potential to enhance the 3D perception of autonomous vehicles. However, existing methods rely on a complex cascaded framework with relatively limited information to restore 3D scenes, including a dependency on supervision solely on the whole network's output, single-frame input, and the utilization of a small backbone. These challenges, in turn, hinder the optimization of the framework and yield inferior prediction results, particularly concerning smaller and long-tailed objects. To address these issues, we propose MonoOcc. In particular, we (i) improve the monocular occupancy prediction framework by proposing an auxiliary semantic loss as supervision to the shallow layers of the framework and an image-conditioned cross-attention module to refine voxel features with visual clues, and (ii) employ a distillation module that transfers temporal information and richer knowledge from a larger image backbone to the monocular semantic occupancy prediction framework with low cost of hardware. With these advantages, our method yields state-of-the-art performance on the camera-based SemanticKITTI Scene Completion benchmark. Codes and models can be accessed at https://github.com/ucaszyp/MonoOcc |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_08766 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MonoOcc: Digging into Monocular Semantic Occupancy Prediction Zheng, Yupeng Li, Xiang Li, Pengfei Zheng, Yuhang Jin, Bu Zhong, Chengliang Long, Xiaoxiao Zhao, Hao Zhang, Qichao Computer Vision and Pattern Recognition Monocular Semantic Occupancy Prediction aims to infer the complete 3D geometry and semantic information of scenes from only 2D images. It has garnered significant attention, particularly due to its potential to enhance the 3D perception of autonomous vehicles. However, existing methods rely on a complex cascaded framework with relatively limited information to restore 3D scenes, including a dependency on supervision solely on the whole network's output, single-frame input, and the utilization of a small backbone. These challenges, in turn, hinder the optimization of the framework and yield inferior prediction results, particularly concerning smaller and long-tailed objects. To address these issues, we propose MonoOcc. In particular, we (i) improve the monocular occupancy prediction framework by proposing an auxiliary semantic loss as supervision to the shallow layers of the framework and an image-conditioned cross-attention module to refine voxel features with visual clues, and (ii) employ a distillation module that transfers temporal information and richer knowledge from a larger image backbone to the monocular semantic occupancy prediction framework with low cost of hardware. With these advantages, our method yields state-of-the-art performance on the camera-based SemanticKITTI Scene Completion benchmark. Codes and models can be accessed at https://github.com/ucaszyp/MonoOcc |
| title | MonoOcc: Digging into Monocular Semantic Occupancy Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.08766 |