HyperNeRFGAN: Hypernetwork approach to 3D NeRF GAN
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913476037836800 |
|---|---|
| author | Kania, Adam Kasymov, Artur Kościukiewicz, Jakub Górak, Artur Mazur, Marcin Zięba, Maciej Spurek, Przemysław |
| author_facet | Kania, Adam Kasymov, Artur Kościukiewicz, Jakub Górak, Artur Mazur, Marcin Zięba, Maciej Spurek, Przemysław |
| contents | The recent surge in popularity of deep generative models for 3D objects has highlighted the need for more efficient training methods, particularly given the difficulties associated with training with conventional 3D representations, such as voxels or point clouds. Neural Radiance Fields (NeRFs), which provide the current benchmark in terms of quality for the generation of novel views of complex 3D scenes from a limited set of 2D images, represent a promising solution to this challenge. However, the training of these models requires the knowledge of the respective camera positions from which the images were viewed. In this paper, we overcome this limitation by introducing HyperNeRFGAN, a Generative Adversarial Network (GAN) architecture employing a hypernetwork paradigm to transform a Gaussian noise into the weights of a NeRF architecture that does not utilize viewing directions in its training phase. Consequently, as evidenced by the findings of our experimental study, the proposed model, despite its notable simplicity in comparison to existing state-of-the-art alternatives, demonstrates superior performance on a diverse range of image datasets where camera position estimation is challenging, particularly in the context of medical data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_11631 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | HyperNeRFGAN: Hypernetwork approach to 3D NeRF GAN Kania, Adam Kasymov, Artur Kościukiewicz, Jakub Górak, Artur Mazur, Marcin Zięba, Maciej Spurek, Przemysław Computer Vision and Pattern Recognition The recent surge in popularity of deep generative models for 3D objects has highlighted the need for more efficient training methods, particularly given the difficulties associated with training with conventional 3D representations, such as voxels or point clouds. Neural Radiance Fields (NeRFs), which provide the current benchmark in terms of quality for the generation of novel views of complex 3D scenes from a limited set of 2D images, represent a promising solution to this challenge. However, the training of these models requires the knowledge of the respective camera positions from which the images were viewed. In this paper, we overcome this limitation by introducing HyperNeRFGAN, a Generative Adversarial Network (GAN) architecture employing a hypernetwork paradigm to transform a Gaussian noise into the weights of a NeRF architecture that does not utilize viewing directions in its training phase. Consequently, as evidenced by the findings of our experimental study, the proposed model, despite its notable simplicity in comparison to existing state-of-the-art alternatives, demonstrates superior performance on a diverse range of image datasets where camera position estimation is challenging, particularly in the context of medical data. |
| title | HyperNeRFGAN: Hypernetwork approach to 3D NeRF GAN |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2301.11631 |