Textured Gaussians for Enhanced 3D Scene Appearance Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914076934799360 |
|---|---|
| author | Chao, Brian Tseng, Hung-Yu Porzi, Lorenzo Gao, Chen Li, Tuotuo Li, Qinbo Saraf, Ayush Huang, Jia-Bin Kopf, Johannes Wetzstein, Gordon Kim, Changil |
| author_facet | Chao, Brian Tseng, Hung-Yu Porzi, Lorenzo Gao, Chen Li, Tuotuo Li, Qinbo Saraf, Ayush Huang, Jia-Bin Kopf, Johannes Wetzstein, Gordon Kim, Changil |
| contents | 3D Gaussian Splatting (3DGS) has recently emerged as a state-of-the-art 3D reconstruction and rendering technique due to its high-quality results and fast training and rendering time. However, pixels covered by the same Gaussian are always shaded in the same color up to a Gaussian falloff scaling factor. Furthermore, the finest geometric detail any individual Gaussian can represent is a simple ellipsoid. These properties of 3DGS greatly limit the expressivity of individual Gaussian primitives. To address these issues, we draw inspiration from texture and alpha mapping in traditional graphics and integrate it with 3DGS. Specifically, we propose a new generalized Gaussian appearance representation that augments each Gaussian with alpha~(A), RGB, or RGBA texture maps to model spatially varying color and opacity across the extent of each Gaussian. As such, each Gaussian can represent a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting. Surprisingly, we found that the expressivity of Gaussians can be greatly improved by using alpha-only texture maps, and further augmenting Gaussians with RGB texture maps achieves the highest expressivity. We validate our method on a wide variety of standard benchmark datasets and our own custom captures at both the object and scene levels. We demonstrate image quality improvements over existing methods while using a similar or lower number of Gaussians. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18625 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Textured Gaussians for Enhanced 3D Scene Appearance Modeling Chao, Brian Tseng, Hung-Yu Porzi, Lorenzo Gao, Chen Li, Tuotuo Li, Qinbo Saraf, Ayush Huang, Jia-Bin Kopf, Johannes Wetzstein, Gordon Kim, Changil Computer Vision and Pattern Recognition Artificial Intelligence Graphics Image and Video Processing 3D Gaussian Splatting (3DGS) has recently emerged as a state-of-the-art 3D reconstruction and rendering technique due to its high-quality results and fast training and rendering time. However, pixels covered by the same Gaussian are always shaded in the same color up to a Gaussian falloff scaling factor. Furthermore, the finest geometric detail any individual Gaussian can represent is a simple ellipsoid. These properties of 3DGS greatly limit the expressivity of individual Gaussian primitives. To address these issues, we draw inspiration from texture and alpha mapping in traditional graphics and integrate it with 3DGS. Specifically, we propose a new generalized Gaussian appearance representation that augments each Gaussian with alpha~(A), RGB, or RGBA texture maps to model spatially varying color and opacity across the extent of each Gaussian. As such, each Gaussian can represent a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting. Surprisingly, we found that the expressivity of Gaussians can be greatly improved by using alpha-only texture maps, and further augmenting Gaussians with RGB texture maps achieves the highest expressivity. We validate our method on a wide variety of standard benchmark datasets and our own custom captures at both the object and scene levels. We demonstrate image quality improvements over existing methods while using a similar or lower number of Gaussians. |
| title | Textured Gaussians for Enhanced 3D Scene Appearance Modeling |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics Image and Video Processing |
| url | https://arxiv.org/abs/2411.18625 |