DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910976375259136 |
|---|---|
| author | Zhang, Jiaxu Zeng, Xianfang Chen, Xin Zuo, Wei Yu, Gang Lin, Guosheng Tu, Zhigang |
| author_facet | Zhang, Jiaxu Zeng, Xianfang Chen, Xin Zuo, Wei Yu, Gang Lin, Guosheng Tu, Zhigang |
| contents | This paper presents DreamDance, a novel character art animation framework capable of producing stable, consistent character and scene motion conditioned on precise camera trajectories. To achieve this, we re-formulate the animation task as two inpainting-based steps: Camera-aware Scene Inpainting and Pose-aware Video Inpainting. The first step leverages a pre-trained image inpainting model to generate multi-view scene images from the reference art and optimizes a stable large-scale Gaussian field, which enables coarse background video rendering with camera trajectories. However, the rendered video is rough and only conveys scene motion. To resolve this, the second step trains a pose-aware video inpainting model that injects the dynamic character into the scene video while enhancing background quality. Specifically, this model is a DiT-based video generation model with a gating strategy that adaptively integrates the character's appearance and pose information into the base background video. Through extensive experiments, we demonstrate the effectiveness and generalizability of DreamDance, producing high-quality and consistent character animations with remarkable camera dynamics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24733 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds Zhang, Jiaxu Zeng, Xianfang Chen, Xin Zuo, Wei Yu, Gang Lin, Guosheng Tu, Zhigang Computer Vision and Pattern Recognition This paper presents DreamDance, a novel character art animation framework capable of producing stable, consistent character and scene motion conditioned on precise camera trajectories. To achieve this, we re-formulate the animation task as two inpainting-based steps: Camera-aware Scene Inpainting and Pose-aware Video Inpainting. The first step leverages a pre-trained image inpainting model to generate multi-view scene images from the reference art and optimizes a stable large-scale Gaussian field, which enables coarse background video rendering with camera trajectories. However, the rendered video is rough and only conveys scene motion. To resolve this, the second step trains a pose-aware video inpainting model that injects the dynamic character into the scene video while enhancing background quality. Specifically, this model is a DiT-based video generation model with a gating strategy that adaptively integrates the character's appearance and pose information into the base background video. Through extensive experiments, we demonstrate the effectiveness and generalizability of DreamDance, producing high-quality and consistent character animations with remarkable camera dynamics. |
| title | DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.24733 |