Beyond Static Scenes: Camera-controllable Background Generation for Human Motion
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910902425485312 |
|---|---|
| author | Yao, Mingshuai Chen, Mengting Zhou, Qinye Zhang, Yabo Liu, Ming Li, Xiaoming Liu, Shaohui Ju, Chen Xiao, Shuai Liu, Qingwen Lan, Jinsong Zuo, Wangmeng |
| author_facet | Yao, Mingshuai Chen, Mengting Zhou, Qinye Zhang, Yabo Liu, Ming Li, Xiaoming Liu, Shaohui Ju, Chen Xiao, Shuai Liu, Qingwen Lan, Jinsong Zuo, Wangmeng |
| contents | In this paper, we investigate the generation of new video backgrounds given a human foreground video, a camera pose, and a reference scene image. This task presents three key challenges. First, the generated background should precisely follow the camera movements corresponding to the human foreground. Second, as the camera shifts in different directions, newly revealed content should appear seamless and natural. Third, objects within the video frame should maintain consistent textures as the camera moves to ensure visual coherence. To address these challenges, we propose DynaScene, a new framework that uses camera poses extracted from the original video as an explicit control to drive background motion. Specifically, we design a multi-task learning paradigm that incorporates auxiliary tasks, namely background outpainting and scene variation, to enhance the realism of the generated backgrounds. Given the scarcity of suitable data, we constructed a large-scale, high-quality dataset tailored for this task, comprising video foregrounds, reference scene images, and corresponding camera poses. This dataset contains 200K video clips, ten times larger than existing real-world human video datasets, providing a significantly richer and more diverse training resource. Project page: https://yaomingshuai.github.io/Beyond-Static-Scenes.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02004 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Static Scenes: Camera-controllable Background Generation for Human Motion Yao, Mingshuai Chen, Mengting Zhou, Qinye Zhang, Yabo Liu, Ming Li, Xiaoming Liu, Shaohui Ju, Chen Xiao, Shuai Liu, Qingwen Lan, Jinsong Zuo, Wangmeng Graphics In this paper, we investigate the generation of new video backgrounds given a human foreground video, a camera pose, and a reference scene image. This task presents three key challenges. First, the generated background should precisely follow the camera movements corresponding to the human foreground. Second, as the camera shifts in different directions, newly revealed content should appear seamless and natural. Third, objects within the video frame should maintain consistent textures as the camera moves to ensure visual coherence. To address these challenges, we propose DynaScene, a new framework that uses camera poses extracted from the original video as an explicit control to drive background motion. Specifically, we design a multi-task learning paradigm that incorporates auxiliary tasks, namely background outpainting and scene variation, to enhance the realism of the generated backgrounds. Given the scarcity of suitable data, we constructed a large-scale, high-quality dataset tailored for this task, comprising video foregrounds, reference scene images, and corresponding camera poses. This dataset contains 200K video clips, ten times larger than existing real-world human video datasets, providing a significantly richer and more diverse training resource. Project page: https://yaomingshuai.github.io/Beyond-Static-Scenes.github.io/ |
| title | Beyond Static Scenes: Camera-controllable Background Generation for Human Motion |
| topic | Graphics |
| url | https://arxiv.org/abs/2504.02004 |