ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917334649667584 |
|---|---|
| author | Yang, Songlin Wang, Zhe Yang, Xuyi Zhang, Songchun Kong, Xianghao Wu, Taiyi Zhao, Xiaotong Zhang, Ran Zhao, Alan Rao, Anyi |
| author_facet | Yang, Songlin Wang, Zhe Yang, Xuyi Zhang, Songchun Kong, Xianghao Wu, Taiyi Zhao, Xiaotong Zhang, Ran Zhao, Alan Rao, Anyi |
| contents | Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11421 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation Yang, Songlin Wang, Zhe Yang, Xuyi Zhang, Songchun Kong, Xianghao Wu, Taiyi Zhao, Xiaotong Zhang, Ran Zhao, Alan Rao, Anyi Computer Vision and Pattern Recognition Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a "Plan-then-Control" framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent. |
| title | ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.11421 |