JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916279976198144 |
|---|---|
| author | Zheng, Zihan Zhong, Houqiang Hu, Qiang Zhang, Xiaoyun Song, Li Zhang, Ya Wang, Yanfeng |
| author_facet | Zheng, Zihan Zhong, Houqiang Hu, Qiang Zhang, Xiaoyun Song, Li Zhang, Ya Wang, Yanfeng |
| contents | Neural Radiance Field (NeRF) excels in photo-realistically static scenes, inspiring numerous efforts to facilitate volumetric videos. However, rendering dynamic and long-sequence radiance fields remains challenging due to the significant data required to represent volumetric videos. In this paper, we propose a novel end-to-end joint optimization scheme of dynamic NeRF representation and compression, called JointRF, thus achieving significantly improved quality and compression efficiency against the previous methods. Specifically, JointRF employs a compact residual feature grid and a coefficient feature grid to represent the dynamic NeRF. This representation handles large motions without compromising quality while concurrently diminishing temporal redundancy. We also introduce a sequential feature compression subnetwork to further reduce spatial-temporal redundancy. Finally, the representation and compression subnetworks are end-to-end trained combined within the JointRF. Extensive experiments demonstrate that JointRF can achieve superior compression performance across various datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_14452 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression Zheng, Zihan Zhong, Houqiang Hu, Qiang Zhang, Xiaoyun Song, Li Zhang, Ya Wang, Yanfeng Computer Vision and Pattern Recognition Artificial Intelligence Neural Radiance Field (NeRF) excels in photo-realistically static scenes, inspiring numerous efforts to facilitate volumetric videos. However, rendering dynamic and long-sequence radiance fields remains challenging due to the significant data required to represent volumetric videos. In this paper, we propose a novel end-to-end joint optimization scheme of dynamic NeRF representation and compression, called JointRF, thus achieving significantly improved quality and compression efficiency against the previous methods. Specifically, JointRF employs a compact residual feature grid and a coefficient feature grid to represent the dynamic NeRF. This representation handles large motions without compromising quality while concurrently diminishing temporal redundancy. We also introduce a sequential feature compression subnetwork to further reduce spatial-temporal redundancy. Finally, the representation and compression subnetworks are end-to-end trained combined within the JointRF. Extensive experiments demonstrate that JointRF can achieve superior compression performance across various datasets. |
| title | JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2405.14452 |