Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.00437 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912895381536768 |
|---|---|
| author | Song, Qi Li, Chenghong Lin, Haotong Peng, Sida Huang, Rui |
| author_facet | Song, Qi Li, Chenghong Lin, Haotong Peng, Sida Huang, Rui |
| contents | We present a novel approach, termed ADGaussian, for generalizable street scene reconstruction. The proposed method enables high-quality rendering from merely single-view input. Unlike prior Gaussian Splatting methods that primarily focus on geometry refinement, we emphasize the importance of joint optimization of image and depth features for accurate Gaussian prediction. To this end, we first incorporate sparse LiDAR depth as an additional input modality, formulating the Gaussian prediction process as a joint learning framework of visual information and geometric clue. Furthermore, we propose a Multi-modal Feature Matching strategy coupled with a Multi-scale Gaussian Decoding model to enhance the joint refinement of multi-modal features, thereby enabling efficient multi-modal Gaussian learning. Extensive experiments on Waymo and KITTI demonstrate that our ADGaussian achieves state-of-the-art performance and exhibits superior zero-shot generalization capabilities in novel-view shifting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_00437 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ADGaussian: Generalizable Gaussian Splatting for Autonomous Driving via Multi-modal Joint Learning Song, Qi Li, Chenghong Lin, Haotong Peng, Sida Huang, Rui Computer Vision and Pattern Recognition We present a novel approach, termed ADGaussian, for generalizable street scene reconstruction. The proposed method enables high-quality rendering from merely single-view input. Unlike prior Gaussian Splatting methods that primarily focus on geometry refinement, we emphasize the importance of joint optimization of image and depth features for accurate Gaussian prediction. To this end, we first incorporate sparse LiDAR depth as an additional input modality, formulating the Gaussian prediction process as a joint learning framework of visual information and geometric clue. Furthermore, we propose a Multi-modal Feature Matching strategy coupled with a Multi-scale Gaussian Decoding model to enhance the joint refinement of multi-modal features, thereby enabling efficient multi-modal Gaussian learning. Extensive experiments on Waymo and KITTI demonstrate that our ADGaussian achieves state-of-the-art performance and exhibits superior zero-shot generalization capabilities in novel-view shifting. |
| title | ADGaussian: Generalizable Gaussian Splatting for Autonomous Driving via Multi-modal Joint Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.00437 |