DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908510597414912 |
|---|---|
| author | Chen, Yushuo Shao, Ruizhi Pang, Youxin Zhang, Hongwen Wu, Xinyi Wu, Rihui Liu, Yebin |
| author_facet | Chen, Yushuo Shao, Ruizhi Pang, Youxin Zhang, Hongwen Wu, Xinyi Wu, Rihui Liu, Yebin |
| contents | We present a novel framework to reconstruct human avatars from monocular videos. Recent approaches have struggled either to capture the fine-grained dynamic details from the input or to generate plausible details at novel viewpoints, which mainly stem from the limited representational capacity of the avatar model and insufficient observational data. To overcome these challenges, we propose to leverage the advanced video generative model, Human4DiT, to generate the human motions from alternative perspective as an additional supervision signal. This approach not only enriches the details in previously unseen regions but also effectively regularizes the avatar representation to mitigate artifacts. Furthermore, we introduce two complementary strategies to enhance video generation: To ensure consistent reproduction of human motion, we inject the physical identity into the model through video fine-tuning. For higher-resolution outputs with finer details, a patch-based denoising algorithm is employed. Experimental results demonstrate that our method outperforms recent state-of-the-art approaches and validate the effectiveness of our proposed strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_00403 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective Chen, Yushuo Shao, Ruizhi Pang, Youxin Zhang, Hongwen Wu, Xinyi Wu, Rihui Liu, Yebin Computer Vision and Pattern Recognition We present a novel framework to reconstruct human avatars from monocular videos. Recent approaches have struggled either to capture the fine-grained dynamic details from the input or to generate plausible details at novel viewpoints, which mainly stem from the limited representational capacity of the avatar model and insufficient observational data. To overcome these challenges, we propose to leverage the advanced video generative model, Human4DiT, to generate the human motions from alternative perspective as an additional supervision signal. This approach not only enriches the details in previously unseen regions but also effectively regularizes the avatar representation to mitigate artifacts. Furthermore, we introduce two complementary strategies to enhance video generation: To ensure consistent reproduction of human motion, we inject the physical identity into the model through video fine-tuning. For higher-resolution outputs with finer details, a patch-based denoising algorithm is employed. Experimental results demonstrate that our method outperforms recent state-of-the-art approaches and validate the effectiveness of our proposed strategies. |
| title | DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.00403 |