Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.07190 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911198661836800 |
|---|---|
| author | Zhi, Yihao Li, Chenghong Liao, Hongjie Yang, Xihe Sun, Zhengwentai Chang, Jiahao Cun, Xiaodong Feng, Wensen Han, Xiaoguang |
| author_facet | Zhi, Yihao Li, Chenghong Liao, Hongjie Yang, Xihe Sun, Zhengwentai Chang, Jiahao Cun, Xiaodong Feng, Wensen Han, Xiaoguang |
| contents | Recent breakthroughs in video generation, powered by large-scale datasets and diffusion techniques, have shown that video diffusion models can function as implicit 4D novel view synthesizers. Nevertheless, current methods primarily concentrate on redirecting camera trajectory within the front view while struggling to generate 360-degree viewpoint changes. In this paper, we focus on human-centric subdomain and present MV-Performer, an innovative framework for creating synchronized novel view videos from monocular full-body captures. To achieve a 360-degree synthesis, we extensively leverage the MVHumanNet dataset and incorporate an informative condition signal. Specifically, we use the camera-dependent normal maps rendered from oriented partial point clouds, which effectively alleviate the ambiguity between seen and unseen observations. To maintain synchronization in the generated videos, we propose a multi-view human-centric video diffusion model that fuses information from the reference video, partial rendering, and different viewpoints. Additionally, we provide a robust inference procedure for in-the-wild video cases, which greatly mitigates the artifacts induced by imperfect monocular depth estimation. Extensive experiments on three datasets demonstrate our MV-Performer's state-of-the-art effectiveness and robustness, setting a strong model for human-centric 4D novel view synthesis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07190 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis Zhi, Yihao Li, Chenghong Liao, Hongjie Yang, Xihe Sun, Zhengwentai Chang, Jiahao Cun, Xiaodong Feng, Wensen Han, Xiaoguang Computer Vision and Pattern Recognition Recent breakthroughs in video generation, powered by large-scale datasets and diffusion techniques, have shown that video diffusion models can function as implicit 4D novel view synthesizers. Nevertheless, current methods primarily concentrate on redirecting camera trajectory within the front view while struggling to generate 360-degree viewpoint changes. In this paper, we focus on human-centric subdomain and present MV-Performer, an innovative framework for creating synchronized novel view videos from monocular full-body captures. To achieve a 360-degree synthesis, we extensively leverage the MVHumanNet dataset and incorporate an informative condition signal. Specifically, we use the camera-dependent normal maps rendered from oriented partial point clouds, which effectively alleviate the ambiguity between seen and unseen observations. To maintain synchronization in the generated videos, we propose a multi-view human-centric video diffusion model that fuses information from the reference video, partial rendering, and different viewpoints. Additionally, we provide a robust inference procedure for in-the-wild video cases, which greatly mitigates the artifacts induced by imperfect monocular depth estimation. Extensive experiments on three datasets demonstrate our MV-Performer's state-of-the-art effectiveness and robustness, setting a strong model for human-centric 4D novel view synthesis. |
| title | MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.07190 |