HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917670212861952 |
|---|---|
| author | Chen, Qifeng Xie, Rengan Huang, Kai Wang, Qi Zheng, Wenting Li, Rong Huo, Yuchi |
| author_facet | Chen, Qifeng Xie, Rengan Huang, Kai Wang, Qi Zheng, Wenting Li, Rong Huo, Yuchi |
| contents | Recently, implicit neural representation has been widely used to generate animatable human avatars. However, the materials and geometry of those representations are coupled in the neural network and hard to edit, which hinders their application in traditional graphics engines. We present a framework for acquiring human avatars that are attached with high-resolution physically-based material textures and triangular mesh from monocular video. Our method introduces a novel information fusion strategy to combine the information from the monocular video and synthesize virtual multi-view images to tackle the sparsity of the input view. We reconstruct humans as deformable neural implicit surfaces and extract triangle mesh in a well-behaved pose as the initial mesh of the next stage. In addition, we introduce an approach to correct the bias for the boundary and size of the coarse mesh extracted. Finally, we adapt prior knowledge of the latent diffusion model at super-resolution in multi-view to distill the decomposed texture. Experiments show that our approach outperforms previous representations in terms of high fidelity, and this explicit result supports deployment on common renderers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11270 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos Chen, Qifeng Xie, Rengan Huang, Kai Wang, Qi Zheng, Wenting Li, Rong Huo, Yuchi Computer Vision and Pattern Recognition Recently, implicit neural representation has been widely used to generate animatable human avatars. However, the materials and geometry of those representations are coupled in the neural network and hard to edit, which hinders their application in traditional graphics engines. We present a framework for acquiring human avatars that are attached with high-resolution physically-based material textures and triangular mesh from monocular video. Our method introduces a novel information fusion strategy to combine the information from the monocular video and synthesize virtual multi-view images to tackle the sparsity of the input view. We reconstruct humans as deformable neural implicit surfaces and extract triangle mesh in a well-behaved pose as the initial mesh of the next stage. In addition, we introduce an approach to correct the bias for the boundary and size of the coarse mesh extracted. Finally, we adapt prior knowledge of the latent diffusion model at super-resolution in multi-view to distill the decomposed texture. Experiments show that our approach outperforms previous representations in terms of high fidelity, and this explicit result supports deployment on common renderers. |
| title | HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.11270 |