Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.01866 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913823327256576 |
|---|---|
| author | Zhang, Yunxiang Li, Bingxuan Kuznetsov, Alexandr Jindal, Akshay Diolatzis, Stavros Chen, Kenneth Sochenov, Anton Kaplanyan, Anton Sun, Qi |
| author_facet | Zhang, Yunxiang Li, Bingxuan Kuznetsov, Alexandr Jindal, Akshay Diolatzis, Stavros Chen, Kenneth Sochenov, Anton Kaplanyan, Anton Sun, Qi |
| contents | Neural image representations have emerged as a promising approach for encoding and rendering visual data. Combined with learning-based workflows, they demonstrate impressive trade-offs between visual fidelity and memory footprint. Existing methods in this domain, however, often rely on fixed data structures that suboptimally allocate memory or compute-intensive implicit models, hindering their practicality for real-time graphics applications.
Inspired by recent advancements in radiance field rendering, we introduce Image-GS, a content-adaptive image representation based on 2D Gaussians. Leveraging a custom differentiable renderer, Image-GS reconstructs images by adaptively allocating and progressively optimizing a group of anisotropic, colored 2D Gaussians. It achieves a favorable balance between visual fidelity and memory efficiency across a variety of stylized images frequently seen in graphics workflows, especially for those showing non-uniformly distributed features and in low-bitrate regimes. Moreover, it supports hardware-friendly rapid random access for real-time usage, requiring only 0.3K MACs to decode a pixel. Through error-guided progressive optimization, Image-GS naturally constructs a smooth level-of-detail hierarchy. We demonstrate its versatility with several applications, including texture compression, semantics-aware compression, and joint image compression and restoration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_01866 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Image-GS: Content-Adaptive Image Representation via 2D Gaussians Zhang, Yunxiang Li, Bingxuan Kuznetsov, Alexandr Jindal, Akshay Diolatzis, Stavros Chen, Kenneth Sochenov, Anton Kaplanyan, Anton Sun, Qi Computer Vision and Pattern Recognition Graphics Neural image representations have emerged as a promising approach for encoding and rendering visual data. Combined with learning-based workflows, they demonstrate impressive trade-offs between visual fidelity and memory footprint. Existing methods in this domain, however, often rely on fixed data structures that suboptimally allocate memory or compute-intensive implicit models, hindering their practicality for real-time graphics applications. Inspired by recent advancements in radiance field rendering, we introduce Image-GS, a content-adaptive image representation based on 2D Gaussians. Leveraging a custom differentiable renderer, Image-GS reconstructs images by adaptively allocating and progressively optimizing a group of anisotropic, colored 2D Gaussians. It achieves a favorable balance between visual fidelity and memory efficiency across a variety of stylized images frequently seen in graphics workflows, especially for those showing non-uniformly distributed features and in low-bitrate regimes. Moreover, it supports hardware-friendly rapid random access for real-time usage, requiring only 0.3K MACs to decode a pixel. Through error-guided progressive optimization, Image-GS naturally constructs a smooth level-of-detail hierarchy. We demonstrate its versatility with several applications, including texture compression, semantics-aware compression, and joint image compression and restoration. |
| title | Image-GS: Content-Adaptive Image Representation via 2D Gaussians |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2407.01866 |