MatchDiffusion: Training-free Generation of Match-cuts
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915039042076672 |
|---|---|
| author | Pardo, Alejandro Pizzati, Fabio Zhang, Tong Pondaven, Alexander Torr, Philip Perez, Juan Camilo Ghanem, Bernard |
| author_facet | Pardo, Alejandro Pizzati, Fabio Zhang, Tong Pondaven, Alexander Torr, Philip Perez, Juan Camilo Ghanem, Bernard |
| contents | Match-cuts are powerful cinematic tools that create seamless transitions between scenes, delivering strong visual and metaphorical connections. However, crafting match-cuts is a challenging, resource-intensive process requiring deliberate artistic planning. In MatchDiffusion, we present the first training-free method for match-cut generation using text-to-video diffusion models. MatchDiffusion leverages a key property of diffusion models: early denoising steps define the scene's broad structure, while later steps add details. Guided by this insight, MatchDiffusion employs "Joint Diffusion" to initialize generation for two prompts from shared noise, aligning structure and motion. It then applies "Disjoint Diffusion", allowing the videos to diverge and introduce unique details. This approach produces visually coherent videos suited for match-cuts. User studies and metrics demonstrate MatchDiffusion's effectiveness and potential to democratize match-cut creation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18677 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MatchDiffusion: Training-free Generation of Match-cuts Pardo, Alejandro Pizzati, Fabio Zhang, Tong Pondaven, Alexander Torr, Philip Perez, Juan Camilo Ghanem, Bernard Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Match-cuts are powerful cinematic tools that create seamless transitions between scenes, delivering strong visual and metaphorical connections. However, crafting match-cuts is a challenging, resource-intensive process requiring deliberate artistic planning. In MatchDiffusion, we present the first training-free method for match-cut generation using text-to-video diffusion models. MatchDiffusion leverages a key property of diffusion models: early denoising steps define the scene's broad structure, while later steps add details. Guided by this insight, MatchDiffusion employs "Joint Diffusion" to initialize generation for two prompts from shared noise, aligning structure and motion. It then applies "Disjoint Diffusion", allowing the videos to diverge and introduce unique details. This approach produces visually coherent videos suited for match-cuts. User studies and metrics demonstrate MatchDiffusion's effectiveness and potential to democratize match-cut creation. |
| title | MatchDiffusion: Training-free Generation of Match-cuts |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2411.18677 |