HPC: Hierarchical Progressive Coding Framework for Volumetric Video
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917740397199360 |
|---|---|
| author | Zheng, Zihan Zhong, Houqiang Hu, Qiang Zhang, Xiaoyun Song, Li Zhang, Ya Wang, Yanfeng |
| author_facet | Zheng, Zihan Zhong, Houqiang Hu, Qiang Zhang, Xiaoyun Song, Li Zhang, Ya Wang, Yanfeng |
| contents | Volumetric video based on Neural Radiance Field (NeRF) holds vast potential for various 3D applications, but its substantial data volume poses significant challenges for compression and transmission. Current NeRF compression lacks the flexibility to adjust video quality and bitrate within a single model for various network and device capacities. To address these issues, we propose HPC, a novel hierarchical progressive volumetric video coding framework achieving variable bitrate using a single model. Specifically, HPC introduces a hierarchical representation with a multi-resolution residual radiance field to reduce temporal redundancy in long-duration sequences while simultaneously generating various levels of detail. Then, we propose an end-to-end progressive learning approach with a multi-rate-distortion loss function to jointly optimize both hierarchical representation and compression. Our HPC trained only once can realize multiple compression levels, while the current methods need to train multiple fixed-bitrate models for different rate-distortion (RD) tradeoffs. Extensive experiments demonstrate that HPC achieves flexible quality levels with variable bitrate by a single model and exhibits competitive RD performance, even outperforming fixed-bitrate models across various datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_09026 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HPC: Hierarchical Progressive Coding Framework for Volumetric Video Zheng, Zihan Zhong, Houqiang Hu, Qiang Zhang, Xiaoyun Song, Li Zhang, Ya Wang, Yanfeng Computer Vision and Pattern Recognition Machine Learning Multimedia Image and Video Processing Volumetric video based on Neural Radiance Field (NeRF) holds vast potential for various 3D applications, but its substantial data volume poses significant challenges for compression and transmission. Current NeRF compression lacks the flexibility to adjust video quality and bitrate within a single model for various network and device capacities. To address these issues, we propose HPC, a novel hierarchical progressive volumetric video coding framework achieving variable bitrate using a single model. Specifically, HPC introduces a hierarchical representation with a multi-resolution residual radiance field to reduce temporal redundancy in long-duration sequences while simultaneously generating various levels of detail. Then, we propose an end-to-end progressive learning approach with a multi-rate-distortion loss function to jointly optimize both hierarchical representation and compression. Our HPC trained only once can realize multiple compression levels, while the current methods need to train multiple fixed-bitrate models for different rate-distortion (RD) tradeoffs. Extensive experiments demonstrate that HPC achieves flexible quality levels with variable bitrate by a single model and exhibits competitive RD performance, even outperforming fixed-bitrate models across various datasets. |
| title | HPC: Hierarchical Progressive Coding Framework for Volumetric Video |
| topic | Computer Vision and Pattern Recognition Machine Learning Multimedia Image and Video Processing |
| url | https://arxiv.org/abs/2407.09026 |