OmniCam: Unified Multimodal Video Generation via Camera Control
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908299071324160 |
|---|---|
| author | Yang, Xiaoda Xu, Jiayang Luan, Kaixuan Zhan, Xinyu Qiu, Hongshun Shi, Shijun Li, Hao Yang, Shuai Zhang, Li Yu, Checheng Lu, Cewu Yang, Lixin |
| author_facet | Yang, Xiaoda Xu, Jiayang Luan, Kaixuan Zhan, Xinyu Qiu, Hongshun Shi, Shijun Li, Hao Yang, Shuai Zhang, Li Yu, Checheng Lu, Cewu Yang, Lixin |
| contents | Camera control, which achieves diverse visual effects by changing camera position and pose, has attracted widespread attention. However, existing methods face challenges such as complex interaction and limited control capabilities. To address these issues, we present OmniCam, a unified multimodal camera control framework. Leveraging large language models and video diffusion models, OmniCam generates spatio-temporally consistent videos. It supports various combinations of input modalities: the user can provide text or video with expected trajectory as camera path guidance, and image or video as content reference, enabling precise control over camera motion. To facilitate the training of OmniCam, we introduce the OmniTr dataset, which contains a large collection of high-quality long-sequence trajectories, videos, and corresponding descriptions. Experimental results demonstrate that our model achieves state-of-the-art performance in high-quality camera-controlled video generation across various metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02312 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OmniCam: Unified Multimodal Video Generation via Camera Control Yang, Xiaoda Xu, Jiayang Luan, Kaixuan Zhan, Xinyu Qiu, Hongshun Shi, Shijun Li, Hao Yang, Shuai Zhang, Li Yu, Checheng Lu, Cewu Yang, Lixin Computer Vision and Pattern Recognition Artificial Intelligence Camera control, which achieves diverse visual effects by changing camera position and pose, has attracted widespread attention. However, existing methods face challenges such as complex interaction and limited control capabilities. To address these issues, we present OmniCam, a unified multimodal camera control framework. Leveraging large language models and video diffusion models, OmniCam generates spatio-temporally consistent videos. It supports various combinations of input modalities: the user can provide text or video with expected trajectory as camera path guidance, and image or video as content reference, enabling precise control over camera motion. To facilitate the training of OmniCam, we introduce the OmniTr dataset, which contains a large collection of high-quality long-sequence trajectories, videos, and corresponding descriptions. Experimental results demonstrate that our model achieves state-of-the-art performance in high-quality camera-controlled video generation across various metrics. |
| title | OmniCam: Unified Multimodal Video Generation via Camera Control |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2504.02312 |