Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909242173161472 |
|---|---|
| author | Qu, Ziyuan Vengurlekar, Omkar Qadri, Mohamad Zhang, Kevin Kaess, Michael Metzler, Christopher Jayasuriya, Suren Pediredla, Adithya |
| author_facet | Qu, Ziyuan Vengurlekar, Omkar Qadri, Mohamad Zhang, Kevin Kaess, Michael Metzler, Christopher Jayasuriya, Suren Pediredla, Adithya |
| contents | Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS represents a scene as a set of 3D Gaussians with varying opacities and employs a computationally efficient splatting operation along with analytical derivatives to compute the 3D Gaussian parameters given scene images captured from various viewpoints. Unfortunately, capturing surround view ($360^{\circ}$ viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, rooms inside a building, and autonomous navigation. In these restricted baseline imaging scenarios, the GS algorithm suffers from a well-known 'missing cone' problem, which results in poor reconstruction along the depth axis. In this manuscript, we demonstrate that using transient data (from sonars) allows us to address the missing cone problem by sampling high-frequency data along the depth axis. We extend the Gaussian splatting algorithms for two commonly used sonars and propose fusion algorithms that simultaneously utilize RGB camera data and sonar data. Through simulations, emulations, and hardware experiments across various imaging scenarios, we show that the proposed fusion algorithms lead to significantly better novel view synthesis (5 dB improvement in PSNR) and 3D geometry reconstruction (60% lower Chamfer distance). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_04687 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion Qu, Ziyuan Vengurlekar, Omkar Qadri, Mohamad Zhang, Kevin Kaess, Michael Metzler, Christopher Jayasuriya, Suren Pediredla, Adithya Computer Vision and Pattern Recognition Graphics Machine Learning Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS represents a scene as a set of 3D Gaussians with varying opacities and employs a computationally efficient splatting operation along with analytical derivatives to compute the 3D Gaussian parameters given scene images captured from various viewpoints. Unfortunately, capturing surround view ($360^{\circ}$ viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, rooms inside a building, and autonomous navigation. In these restricted baseline imaging scenarios, the GS algorithm suffers from a well-known 'missing cone' problem, which results in poor reconstruction along the depth axis. In this manuscript, we demonstrate that using transient data (from sonars) allows us to address the missing cone problem by sampling high-frequency data along the depth axis. We extend the Gaussian splatting algorithms for two commonly used sonars and propose fusion algorithms that simultaneously utilize RGB camera data and sonar data. Through simulations, emulations, and hardware experiments across various imaging scenarios, we show that the proposed fusion algorithms lead to significantly better novel view synthesis (5 dB improvement in PSNR) and 3D geometry reconstruction (60% lower Chamfer distance). |
| title | Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion |
| topic | Computer Vision and Pattern Recognition Graphics Machine Learning |
| url | https://arxiv.org/abs/2404.04687 |