ZS-SRT: An Efficient Zero-Shot Super-Resolution Training Method for Neural Radiance Fields
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910016910393344 |
|---|---|
| author | Feng, Xiang He, Yongbo Wang, Yubo Wang, Chengkai Kuang, Zhenzhong Ding, Jiajun Qin, Feiwei Yu, Jun Fan, Jianping |
| author_facet | Feng, Xiang He, Yongbo Wang, Yubo Wang, Chengkai Kuang, Zhenzhong Ding, Jiajun Qin, Feiwei Yu, Jun Fan, Jianping |
| contents | Neural Radiance Fields (NeRF) have achieved great success in the task of synthesizing novel views that preserve the same resolution as the training views. However, it is challenging for NeRF to synthesize high-quality high-resolution novel views with low-resolution training data. To solve this problem, we propose a zero-shot super-resolution training framework for NeRF. This framework aims to guide the NeRF model to synthesize high-resolution novel views via single-scene internal learning rather than requiring any external high-resolution training data. Our approach consists of two stages. First, we learn a scene-specific degradation mapping by performing internal learning on a pretrained low-resolution coarse NeRF. Second, we optimize a super-resolution fine NeRF by conducting inverse rendering with our mapping function so as to backpropagate the gradients from low-resolution 2D space into the super-resolution 3D sampling space. Then, we further introduce a temporal ensemble strategy in the inference phase to compensate for the scene estimation errors. Our method is featured on two points: (1) it does not consume high-resolution views or additional scene data to train super-resolution NeRF; (2) it can speed up the training process by adopting a coarse-to-fine strategy. By conducting extensive experiments on public datasets, we have qualitatively and quantitatively demonstrated the effectiveness of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_12122 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ZS-SRT: An Efficient Zero-Shot Super-Resolution Training Method for Neural Radiance Fields Feng, Xiang He, Yongbo Wang, Yubo Wang, Chengkai Kuang, Zhenzhong Ding, Jiajun Qin, Feiwei Yu, Jun Fan, Jianping Computer Vision and Pattern Recognition Graphics Neural Radiance Fields (NeRF) have achieved great success in the task of synthesizing novel views that preserve the same resolution as the training views. However, it is challenging for NeRF to synthesize high-quality high-resolution novel views with low-resolution training data. To solve this problem, we propose a zero-shot super-resolution training framework for NeRF. This framework aims to guide the NeRF model to synthesize high-resolution novel views via single-scene internal learning rather than requiring any external high-resolution training data. Our approach consists of two stages. First, we learn a scene-specific degradation mapping by performing internal learning on a pretrained low-resolution coarse NeRF. Second, we optimize a super-resolution fine NeRF by conducting inverse rendering with our mapping function so as to backpropagate the gradients from low-resolution 2D space into the super-resolution 3D sampling space. Then, we further introduce a temporal ensemble strategy in the inference phase to compensate for the scene estimation errors. Our method is featured on two points: (1) it does not consume high-resolution views or additional scene data to train super-resolution NeRF; (2) it can speed up the training process by adopting a coarse-to-fine strategy. By conducting extensive experiments on public datasets, we have qualitatively and quantitatively demonstrated the effectiveness of our method. |
| title | ZS-SRT: An Efficient Zero-Shot Super-Resolution Training Method for Neural Radiance Fields |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2312.12122 |