VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914868896989184 |
|---|---|
| author | Hwang, Sungwon Kim, Min-Jung Kang, Taewoong Kang, Jayeon Choo, Jaegul |
| author_facet | Hwang, Sungwon Kim, Min-Jung Kang, Taewoong Kang, Jayeon Choo, Jaegul |
| contents | Neural rendering-based urban scene reconstruction methods commonly rely on images collected from driving vehicles with cameras facing and moving forward. Although these methods can successfully synthesize from views similar to training camera trajectory, directing the novel view outside the training camera distribution does not guarantee on-par performance. In this paper, we tackle the Extrapolated View Synthesis (EVS) problem by evaluating the reconstructions on views such as looking left, right or downwards with respect to training camera distributions. To improve rendering quality for EVS, we initialize our model by constructing dense LiDAR map, and propose to leverage prior scene knowledge such as surface normal estimator and large-scale diffusion model. Qualitative and quantitative comparisons demonstrate the effectiveness of our methods on EVS. To the best of our knowledge, we are the first to address the EVS problem in urban scene reconstruction. Link to our project page: https://vegs3d.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02945 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors Hwang, Sungwon Kim, Min-Jung Kang, Taewoong Kang, Jayeon Choo, Jaegul Computer Vision and Pattern Recognition Neural rendering-based urban scene reconstruction methods commonly rely on images collected from driving vehicles with cameras facing and moving forward. Although these methods can successfully synthesize from views similar to training camera trajectory, directing the novel view outside the training camera distribution does not guarantee on-par performance. In this paper, we tackle the Extrapolated View Synthesis (EVS) problem by evaluating the reconstructions on views such as looking left, right or downwards with respect to training camera distributions. To improve rendering quality for EVS, we initialize our model by constructing dense LiDAR map, and propose to leverage prior scene knowledge such as surface normal estimator and large-scale diffusion model. Qualitative and quantitative comparisons demonstrate the effectiveness of our methods on EVS. To the best of our knowledge, we are the first to address the EVS problem in urban scene reconstruction. Link to our project page: https://vegs3d.github.io/. |
| title | VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.02945 |