IDOL: Instant Photorealistic 3D Human Creation from a Single Image
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916661543567360 |
|---|---|
| author | Zhuang, Yiyu Lv, Jiaxi Wen, Hao Shuai, Qing Zeng, Ailing Zhu, Hao Chen, Shifeng Yang, Yujiu Cao, Xun Liu, Wei |
| author_facet | Zhuang, Yiyu Lv, Jiaxi Wen, Hao Shuai, Qing Zeng, Ailing Zhu, Hao Chen, Shifeng Yang, Yujiu Cao, Xun Liu, Wei |
| contents | Creating a high-fidelity, animatable 3D full-body avatar from a single image is a challenging task due to the diverse appearance and poses of humans and the limited availability of high-quality training data. To achieve fast and high-quality human reconstruction, this work rethinks the task from the perspectives of dataset, model, and representation. First, we introduce a large-scale HUman-centric GEnerated dataset, HuGe100K, consisting of 100K diverse, photorealistic sets of human images. Each set contains 24-view frames in specific human poses, generated using a pose-controllable image-to-multi-view model. Next, leveraging the diversity in views, poses, and appearances within HuGe100K, we develop a scalable feed-forward transformer model to predict a 3D human Gaussian representation in a uniform space from a given human image. This model is trained to disentangle human pose, body shape, clothing geometry, and texture. The estimated Gaussians can be animated without post-processing. We conduct comprehensive experiments to validate the effectiveness of the proposed dataset and method. Our model demonstrates the ability to efficiently reconstruct photorealistic humans at 1K resolution from a single input image using a single GPU instantly. Additionally, it seamlessly supports various applications, as well as shape and texture editing tasks. Project page: https://yiyuzhuang.github.io/IDOL/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_14963 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | IDOL: Instant Photorealistic 3D Human Creation from a Single Image Zhuang, Yiyu Lv, Jiaxi Wen, Hao Shuai, Qing Zeng, Ailing Zhu, Hao Chen, Shifeng Yang, Yujiu Cao, Xun Liu, Wei Computer Vision and Pattern Recognition Graphics Machine Learning 68U05, 68T07, 68T45 I.3.7; I.2.10; I.2.6 Creating a high-fidelity, animatable 3D full-body avatar from a single image is a challenging task due to the diverse appearance and poses of humans and the limited availability of high-quality training data. To achieve fast and high-quality human reconstruction, this work rethinks the task from the perspectives of dataset, model, and representation. First, we introduce a large-scale HUman-centric GEnerated dataset, HuGe100K, consisting of 100K diverse, photorealistic sets of human images. Each set contains 24-view frames in specific human poses, generated using a pose-controllable image-to-multi-view model. Next, leveraging the diversity in views, poses, and appearances within HuGe100K, we develop a scalable feed-forward transformer model to predict a 3D human Gaussian representation in a uniform space from a given human image. This model is trained to disentangle human pose, body shape, clothing geometry, and texture. The estimated Gaussians can be animated without post-processing. We conduct comprehensive experiments to validate the effectiveness of the proposed dataset and method. Our model demonstrates the ability to efficiently reconstruct photorealistic humans at 1K resolution from a single input image using a single GPU instantly. Additionally, it seamlessly supports various applications, as well as shape and texture editing tasks. Project page: https://yiyuzhuang.github.io/IDOL/. |
| title | IDOL: Instant Photorealistic 3D Human Creation from a Single Image |
| topic | Computer Vision and Pattern Recognition Graphics Machine Learning 68U05, 68T07, 68T45 I.3.7; I.2.10; I.2.6 |
| url | https://arxiv.org/abs/2412.14963 |