SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910616088739840 |
|---|---|
| author | Zhai, Hongjia Zhang, Xiyu Zhao, Boming Li, Hai He, Yijia Cui, Zhaopeng Bao, Hujun Zhang, Guofeng |
| author_facet | Zhai, Hongjia Zhang, Xiyu Zhao, Boming Li, Hai He, Yijia Cui, Zhaopeng Bao, Hujun Zhang, Guofeng |
| contents | Visual localization plays an important role in the applications of Augmented Reality (AR), which enable AR devices to obtain their 6-DoF pose in the pre-build map in order to render virtual content in real scenes. However, most existing approaches can not perform novel view rendering and require large storage capacities for maps. To overcome these limitations, we propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Specifically, our approach leverages 3D Gaussian primitives as the scene representation. To ensure precise 2D-3D correspondences for pose estimation, we develop an unbiased 3D scene-specific descriptor decoder for Gaussian primitives, distilled from a constructed feature volume. Additionally, we introduce a salient 3D landmark selection algorithm that selects a suitable primitive subset based on the saliency score for localization. We further regularize key Gaussian primitives to prevent anisotropic effects, which also improves localization performance. Extensive experiments on two widely used datasets demonstrate that our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches. Project page: \href{https://zju3dv.github.io/splatloc}{https://zju3dv.github.io/splatloc}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14067 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality Zhai, Hongjia Zhang, Xiyu Zhao, Boming Li, Hai He, Yijia Cui, Zhaopeng Bao, Hujun Zhang, Guofeng Computer Vision and Pattern Recognition Visual localization plays an important role in the applications of Augmented Reality (AR), which enable AR devices to obtain their 6-DoF pose in the pre-build map in order to render virtual content in real scenes. However, most existing approaches can not perform novel view rendering and require large storage capacities for maps. To overcome these limitations, we propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Specifically, our approach leverages 3D Gaussian primitives as the scene representation. To ensure precise 2D-3D correspondences for pose estimation, we develop an unbiased 3D scene-specific descriptor decoder for Gaussian primitives, distilled from a constructed feature volume. Additionally, we introduce a salient 3D landmark selection algorithm that selects a suitable primitive subset based on the saliency score for localization. We further regularize key Gaussian primitives to prevent anisotropic effects, which also improves localization performance. Extensive experiments on two widely used datasets demonstrate that our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches. Project page: \href{https://zju3dv.github.io/splatloc}{https://zju3dv.github.io/splatloc}. |
| title | SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.14067 |