Learning 3D-Aware GANs from Unposed Images with Template Feature Field
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929515587960832 |
|---|---|
| author | Chen, Xinya Guo, Hanlei Bin, Yanrui Zhang, Shangzhan Yang, Yuanbo Wang, Yue Shen, Yujun Liao, Yiyi |
| author_facet | Chen, Xinya Guo, Hanlei Bin, Yanrui Zhang, Shangzhan Yang, Yuanbo Wang, Yue Shen, Yujun Liao, Yiyi |
| contents | Collecting accurate camera poses of training images has been shown to well serve the learning of 3D-aware generative adversarial networks (GANs) yet can be quite expensive in practice. This work targets learning 3D-aware GANs from unposed images, for which we propose to perform on-the-fly pose estimation of training images with a learned template feature field (TeFF). Concretely, in addition to a generative radiance field as in previous approaches, we ask the generator to also learn a field from 2D semantic features while sharing the density from the radiance field. Such a framework allows us to acquire a canonical 3D feature template leveraging the dataset mean discovered by the generative model, and further efficiently estimate the pose parameters on real data. Experimental results on various challenging datasets demonstrate the superiority of our approach over state-of-the-art alternatives from both the qualitative and the quantitative perspectives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_05705 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning 3D-Aware GANs from Unposed Images with Template Feature Field Chen, Xinya Guo, Hanlei Bin, Yanrui Zhang, Shangzhan Yang, Yuanbo Wang, Yue Shen, Yujun Liao, Yiyi Computer Vision and Pattern Recognition Collecting accurate camera poses of training images has been shown to well serve the learning of 3D-aware generative adversarial networks (GANs) yet can be quite expensive in practice. This work targets learning 3D-aware GANs from unposed images, for which we propose to perform on-the-fly pose estimation of training images with a learned template feature field (TeFF). Concretely, in addition to a generative radiance field as in previous approaches, we ask the generator to also learn a field from 2D semantic features while sharing the density from the radiance field. Such a framework allows us to acquire a canonical 3D feature template leveraging the dataset mean discovered by the generative model, and further efficiently estimate the pose parameters on real data. Experimental results on various challenging datasets demonstrate the superiority of our approach over state-of-the-art alternatives from both the qualitative and the quantitative perspectives. |
| title | Learning 3D-Aware GANs from Unposed Images with Template Feature Field |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.05705 |