How Animals Dance (When You're Not Looking)
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917101716897792 |
|---|---|
| author | Wang, Xiaojuan Holynski, Aleksander Curless, Brian Kemelmacher, Ira Seitz, Steve |
| author_facet | Wang, Xiaojuan Holynski, Aleksander Curless, Brian Kemelmacher, Ira Seitz, Steve |
| contents | We present a framework for generating music-synchronized, choreography aware animal dance videos. Our framework introduces choreography patterns -- structured sequences of motion beats that define the long-range structure of a dance -- as a novel high-level control signal for dance video generation. These patterns can be automatically estimated from human dance videos. Starting from a few keyframes representing distinct animal poses, generated via text-to-image prompting or GPT-4o, we formulate dance synthesis as a graph optimization problem that seeks the optimal keyframe structure to satisfy a specified choreography pattern of beats. We also introduce an approach for mirrored pose image generation, essential for capturing symmetry in dance. In-between frames are synthesized using an video diffusion model. With as few as six input keyframes, our method can produce up to 30 seconds dance videos across a wide range of animals and music tracks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23738 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How Animals Dance (When You're Not Looking) Wang, Xiaojuan Holynski, Aleksander Curless, Brian Kemelmacher, Ira Seitz, Steve Computer Vision and Pattern Recognition Graphics We present a framework for generating music-synchronized, choreography aware animal dance videos. Our framework introduces choreography patterns -- structured sequences of motion beats that define the long-range structure of a dance -- as a novel high-level control signal for dance video generation. These patterns can be automatically estimated from human dance videos. Starting from a few keyframes representing distinct animal poses, generated via text-to-image prompting or GPT-4o, we formulate dance synthesis as a graph optimization problem that seeks the optimal keyframe structure to satisfy a specified choreography pattern of beats. We also introduce an approach for mirrored pose image generation, essential for capturing symmetry in dance. In-between frames are synthesized using an video diffusion model. With as few as six input keyframes, our method can produce up to 30 seconds dance videos across a wide range of animals and music tracks. |
| title | How Animals Dance (When You're Not Looking) |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2505.23738 |