GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917715994738688 |
|---|---|
| author | Zhang, Xinjie Ge, Xingtong Xu, Tongda He, Dailan Wang, Yan Qin, Hongwei Lu, Guo Geng, Jing Zhang, Jun |
| author_facet | Zhang, Xinjie Ge, Xingtong Xu, Tongda He, Dailan Wang, Yan Qin, Hongwei Lu, Guo Geng, Jing Zhang, Jun |
| contents | Implicit neural representations (INRs) recently achieved great success in image representation and compression, offering high visual quality and fast rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are available. However, this requirement often hinders their use on low-end devices with limited memory. In response, we propose a groundbreaking paradigm of image representation and compression by 2D Gaussian Splatting, named GaussianImage. We first introduce 2D Gaussian to represent the image, where each Gaussian has 8 parameters including position, covariance and color. Subsequently, we unveil a novel rendering algorithm based on accumulated summation. Remarkably, our method with a minimum of 3$\times$ lower GPU memory usage and 5$\times$ faster fitting time not only rivals INRs (e.g., WIRE, I-NGP) in representation performance, but also delivers a faster rendering speed of 1500-2000 FPS regardless of parameter size. Furthermore, we integrate existing vector quantization technique to build an image codec. Experimental results demonstrate that our codec attains rate-distortion performance comparable to compression-based INRs such as COIN and COIN++, while facilitating decoding speeds of approximately 2000 FPS. Additionally, preliminary proof of concept shows that our codec surpasses COIN and COIN++ in performance when using partial bits-back coding. Code is available at https://github.com/Xinjie-Q/GaussianImage. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_08551 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting Zhang, Xinjie Ge, Xingtong Xu, Tongda He, Dailan Wang, Yan Qin, Hongwei Lu, Guo Geng, Jing Zhang, Jun Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Multimedia Implicit neural representations (INRs) recently achieved great success in image representation and compression, offering high visual quality and fast rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are available. However, this requirement often hinders their use on low-end devices with limited memory. In response, we propose a groundbreaking paradigm of image representation and compression by 2D Gaussian Splatting, named GaussianImage. We first introduce 2D Gaussian to represent the image, where each Gaussian has 8 parameters including position, covariance and color. Subsequently, we unveil a novel rendering algorithm based on accumulated summation. Remarkably, our method with a minimum of 3$\times$ lower GPU memory usage and 5$\times$ faster fitting time not only rivals INRs (e.g., WIRE, I-NGP) in representation performance, but also delivers a faster rendering speed of 1500-2000 FPS regardless of parameter size. Furthermore, we integrate existing vector quantization technique to build an image codec. Experimental results demonstrate that our codec attains rate-distortion performance comparable to compression-based INRs such as COIN and COIN++, while facilitating decoding speeds of approximately 2000 FPS. Additionally, preliminary proof of concept shows that our codec surpasses COIN and COIN++ in performance when using partial bits-back coding. Code is available at https://github.com/Xinjie-Q/GaussianImage. |
| title | GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting |
| topic | Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2403.08551 |