URAvatar: Universal Relightable Gaussian Codec Avatars
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915001167511552 |
|---|---|
| author | Li, Junxuan Cao, Chen Schwartz, Gabriel Khirodkar, Rawal Richardt, Christian Simon, Tomas Sheikh, Yaser Saito, Shunsuke |
| author_facet | Li, Junxuan Cao, Chen Schwartz, Gabriel Khirodkar, Rawal Richardt, Christian Simon, Tomas Sheikh, Yaser Saito, Shunsuke |
| contents | We present a new approach to creating photorealistic and relightable head avatars from a phone scan with unknown illumination. The reconstructed avatars can be animated and relit in real time with the global illumination of diverse environments. Unlike existing approaches that estimate parametric reflectance parameters via inverse rendering, our approach directly models learnable radiance transfer that incorporates global light transport in an efficient manner for real-time rendering. However, learning such a complex light transport that can generalize across identities is non-trivial. A phone scan in a single environment lacks sufficient information to infer how the head would appear in general environments. To address this, we build a universal relightable avatar model represented by 3D Gaussians. We train on hundreds of high-quality multi-view human scans with controllable point lights. High-resolution geometric guidance further enhances the reconstruction accuracy and generalization. Once trained, we finetune the pretrained model on a phone scan using inverse rendering to obtain a personalized relightable avatar. Our experiments establish the efficacy of our design, outperforming existing approaches while retaining real-time rendering capability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_24223 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | URAvatar: Universal Relightable Gaussian Codec Avatars Li, Junxuan Cao, Chen Schwartz, Gabriel Khirodkar, Rawal Richardt, Christian Simon, Tomas Sheikh, Yaser Saito, Shunsuke Computer Vision and Pattern Recognition Graphics We present a new approach to creating photorealistic and relightable head avatars from a phone scan with unknown illumination. The reconstructed avatars can be animated and relit in real time with the global illumination of diverse environments. Unlike existing approaches that estimate parametric reflectance parameters via inverse rendering, our approach directly models learnable radiance transfer that incorporates global light transport in an efficient manner for real-time rendering. However, learning such a complex light transport that can generalize across identities is non-trivial. A phone scan in a single environment lacks sufficient information to infer how the head would appear in general environments. To address this, we build a universal relightable avatar model represented by 3D Gaussians. We train on hundreds of high-quality multi-view human scans with controllable point lights. High-resolution geometric guidance further enhances the reconstruction accuracy and generalization. Once trained, we finetune the pretrained model on a phone scan using inverse rendering to obtain a personalized relightable avatar. Our experiments establish the efficacy of our design, outperforming existing approaches while retaining real-time rendering capability. |
| title | URAvatar: Universal Relightable Gaussian Codec Avatars |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2410.24223 |