Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.11647 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909680537698304 |
|---|---|
| author | Bai, Jianhong Xia, Menghan Fu, Xiao Wang, Xintao Mu, Lianrui Cao, Jinwen Liu, Zuozhu Hu, Haoji Bai, Xiang Wan, Pengfei Zhang, Di |
| author_facet | Bai, Jianhong Xia, Menghan Fu, Xiao Wang, Xintao Mu, Lianrui Cao, Jinwen Liu, Zuozhu Hu, Haoji Bai, Xiang Wan, Pengfei Zhang, Di |
| contents | Camera control has been actively studied in text or image conditioned video generation tasks. However, altering camera trajectories of a given video remains under-explored, despite its importance in the field of video creation. It is non-trivial due to the extra constraints of maintaining multiple-frame appearance and dynamic synchronization. To address this, we present ReCamMaster, a camera-controlled generative video re-rendering framework that reproduces the dynamic scene of an input video at novel camera trajectories. The core innovation lies in harnessing the generative capabilities of pre-trained text-to-video models through a simple yet powerful video conditioning mechanism--its capability is often overlooked in current research. To overcome the scarcity of qualified training data, we construct a comprehensive multi-camera synchronized video dataset using Unreal Engine 5, which is carefully curated to follow real-world filming characteristics, covering diverse scenes and camera movements. It helps the model generalize to in-the-wild videos. Lastly, we further improve the robustness to diverse inputs through a meticulously designed training strategy. Extensive experiments show that our method substantially outperforms existing state-of-the-art approaches. Our method also finds promising applications in video stabilization, super-resolution, and outpainting. Our code and dataset are publicly available at: https://github.com/KwaiVGI/ReCamMaster. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11647 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ReCamMaster: Camera-Controlled Generative Rendering from A Single Video Bai, Jianhong Xia, Menghan Fu, Xiao Wang, Xintao Mu, Lianrui Cao, Jinwen Liu, Zuozhu Hu, Haoji Bai, Xiang Wan, Pengfei Zhang, Di Computer Vision and Pattern Recognition Camera control has been actively studied in text or image conditioned video generation tasks. However, altering camera trajectories of a given video remains under-explored, despite its importance in the field of video creation. It is non-trivial due to the extra constraints of maintaining multiple-frame appearance and dynamic synchronization. To address this, we present ReCamMaster, a camera-controlled generative video re-rendering framework that reproduces the dynamic scene of an input video at novel camera trajectories. The core innovation lies in harnessing the generative capabilities of pre-trained text-to-video models through a simple yet powerful video conditioning mechanism--its capability is often overlooked in current research. To overcome the scarcity of qualified training data, we construct a comprehensive multi-camera synchronized video dataset using Unreal Engine 5, which is carefully curated to follow real-world filming characteristics, covering diverse scenes and camera movements. It helps the model generalize to in-the-wild videos. Lastly, we further improve the robustness to diverse inputs through a meticulously designed training strategy. Extensive experiments show that our method substantially outperforms existing state-of-the-art approaches. Our method also finds promising applications in video stabilization, super-resolution, and outpainting. Our code and dataset are publicly available at: https://github.com/KwaiVGI/ReCamMaster. |
| title | ReCamMaster: Camera-Controlled Generative Rendering from A Single Video |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.11647 |