End-to-End Rate-Distortion Optimized 3D Gaussian Representation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913555594346496 |
|---|---|
| author | Wang, Henan Zhu, Hanxin He, Tianyu Feng, Runsen Deng, Jiajun Bian, Jiang Chen, Zhibo |
| author_facet | Wang, Henan Zhu, Hanxin He, Tianyu Feng, Runsen Deng, Jiajun Bian, Jiang Chen, Zhibo |
| contents | 3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering. However, the substantial storage overhead of 3DGS significantly impedes its practical applications. In this work, we formulate the compact 3D Gaussian learning as an end-to-end Rate-Distortion Optimization (RDO) problem and propose RDO-Gaussian that can achieve flexible and continuous rate control. RDO-Gaussian addresses two main issues that exist in current schemes: 1) Different from prior endeavors that minimize the rate under the fixed distortion, we introduce dynamic pruning and entropy-constrained vector quantization (ECVQ) that optimize the rate and distortion at the same time. 2) Previous works treat the colors of each Gaussian equally, while we model the colors of different regions and materials with learnable numbers of parameters. We verify our method on both real and synthetic scenes, showcasing that RDO-Gaussian greatly reduces the size of 3D Gaussian over 40x, and surpasses existing methods in rate-distortion performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01597 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | End-to-End Rate-Distortion Optimized 3D Gaussian Representation Wang, Henan Zhu, Hanxin He, Tianyu Feng, Runsen Deng, Jiajun Bian, Jiang Chen, Zhibo Computer Vision and Pattern Recognition Graphics 3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering. However, the substantial storage overhead of 3DGS significantly impedes its practical applications. In this work, we formulate the compact 3D Gaussian learning as an end-to-end Rate-Distortion Optimization (RDO) problem and propose RDO-Gaussian that can achieve flexible and continuous rate control. RDO-Gaussian addresses two main issues that exist in current schemes: 1) Different from prior endeavors that minimize the rate under the fixed distortion, we introduce dynamic pruning and entropy-constrained vector quantization (ECVQ) that optimize the rate and distortion at the same time. 2) Previous works treat the colors of each Gaussian equally, while we model the colors of different regions and materials with learnable numbers of parameters. We verify our method on both real and synthetic scenes, showcasing that RDO-Gaussian greatly reduces the size of 3D Gaussian over 40x, and surpasses existing methods in rate-distortion performance. |
| title | End-to-End Rate-Distortion Optimized 3D Gaussian Representation |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2406.01597 |