Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.05310 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910595229417472 |
|---|---|
| author | Liu, Jianheng Zheng, Chunran Wan, Yunfei Wang, Bowen Cai, Yixi Zhang, Fu |
| author_facet | Liu, Jianheng Zheng, Chunran Wan, Yunfei Wang, Bowen Cai, Yixi Zhang, Fu |
| contents | This paper presents a unified surface reconstruction and rendering framework for LiDAR-visual systems, integrating Neural Radiance Fields (NeRF) and Neural Distance Fields (NDF) to recover both appearance and structural information from posed images and point clouds. We address the structural visible gap between NeRF and NDF by utilizing a visible-aware occupancy map to classify space into the free, occupied, visible unknown, and background regions. This classification facilitates the recovery of a complete appearance and structure of the scene. We unify the training of the NDF and NeRF using a spatial-varying scale SDF-to-density transformation for levels of detail for both structure and appearance. The proposed method leverages the learned NDF for structure-aware NeRF training by an adaptive sphere tracing sampling strategy for accurate structure rendering. In return, NeRF further refines structural in recovering missing or fuzzy structures in the NDF. Extensive experiments demonstrate the superior quality and versatility of the proposed method across various scenarios. To benefit the community, the codes will be released at \url{https://github.com/hku-mars/M2Mapping}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_05310 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Neural Surface Reconstruction and Rendering for LiDAR-Visual Systems Liu, Jianheng Zheng, Chunran Wan, Yunfei Wang, Bowen Cai, Yixi Zhang, Fu Robotics Computer Vision and Pattern Recognition This paper presents a unified surface reconstruction and rendering framework for LiDAR-visual systems, integrating Neural Radiance Fields (NeRF) and Neural Distance Fields (NDF) to recover both appearance and structural information from posed images and point clouds. We address the structural visible gap between NeRF and NDF by utilizing a visible-aware occupancy map to classify space into the free, occupied, visible unknown, and background regions. This classification facilitates the recovery of a complete appearance and structure of the scene. We unify the training of the NDF and NeRF using a spatial-varying scale SDF-to-density transformation for levels of detail for both structure and appearance. The proposed method leverages the learned NDF for structure-aware NeRF training by an adaptive sphere tracing sampling strategy for accurate structure rendering. In return, NeRF further refines structural in recovering missing or fuzzy structures in the NDF. Extensive experiments demonstrate the superior quality and versatility of the proposed method across various scenarios. To benefit the community, the codes will be released at \url{https://github.com/hku-mars/M2Mapping}. |
| title | Neural Surface Reconstruction and Rendering for LiDAR-Visual Systems |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.05310 |