MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929299648413696 |
|---|---|
| author | Comas-Massagué, Armand Qiu, Di Chai, Menglei Bühler, Marcel Raj, Amit Gao, Ruiqi Xu, Qiangeng Matthews, Mark Gotardo, Paulo Camps, Octavia Orts-Escolano, Sergio Beeler, Thabo |
| author_facet | Comas-Massagué, Armand Qiu, Di Chai, Menglei Bühler, Marcel Raj, Amit Gao, Ruiqi Xu, Qiangeng Matthews, Mark Gotardo, Paulo Camps, Octavia Orts-Escolano, Sergio Beeler, Thabo |
| contents | We introduce a novel framework for 3D human avatar generation and personalization, leveraging text prompts to enhance user engagement and customization. Central to our approach are key innovations aimed at overcoming the challenges in photo-realistic avatar synthesis. Firstly, we utilize a conditional Neural Radiance Fields (NeRF) model, trained on a large-scale unannotated multi-view dataset, to create a versatile initial solution space that accelerates and diversifies avatar generation. Secondly, we develop a geometric prior, leveraging the capabilities of Text-to-Image Diffusion Models, to ensure superior view invariance and enable direct optimization of avatar geometry. These foundational ideas are complemented by our optimization pipeline built on Variational Score Distillation (VSD), which mitigates texture loss and over-saturation issues. As supported by our extensive experiments, these strategies collectively enable the creation of custom avatars with unparalleled visual quality and better adherence to input text prompts. You can find more results and videos in our website: https://syntec-research.github.io/MagicMirror |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_01296 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space Comas-Massagué, Armand Qiu, Di Chai, Menglei Bühler, Marcel Raj, Amit Gao, Ruiqi Xu, Qiangeng Matthews, Mark Gotardo, Paulo Camps, Octavia Orts-Escolano, Sergio Beeler, Thabo Computer Vision and Pattern Recognition We introduce a novel framework for 3D human avatar generation and personalization, leveraging text prompts to enhance user engagement and customization. Central to our approach are key innovations aimed at overcoming the challenges in photo-realistic avatar synthesis. Firstly, we utilize a conditional Neural Radiance Fields (NeRF) model, trained on a large-scale unannotated multi-view dataset, to create a versatile initial solution space that accelerates and diversifies avatar generation. Secondly, we develop a geometric prior, leveraging the capabilities of Text-to-Image Diffusion Models, to ensure superior view invariance and enable direct optimization of avatar geometry. These foundational ideas are complemented by our optimization pipeline built on Variational Score Distillation (VSD), which mitigates texture loss and over-saturation issues. As supported by our extensive experiments, these strategies collectively enable the creation of custom avatars with unparalleled visual quality and better adherence to input text prompts. You can find more results and videos in our website: https://syntec-research.github.io/MagicMirror |
| title | MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.01296 |