X-Dyna: Expressive Dynamic Human Image Animation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916574302044160 |
|---|---|
| author | Chang, Di Xu, Hongyi Xie, You Gao, Yipeng Kuang, Zhengfei Cai, Shengqu Zhang, Chenxu Song, Guoxian Wang, Chao Shi, Yichun Chen, Zeyuan Zhou, Shijie Luo, Linjie Wetzstein, Gordon Soleymani, Mohammad |
| author_facet | Chang, Di Xu, Hongyi Xie, You Gao, Yipeng Kuang, Zhengfei Cai, Shengqu Zhang, Chenxu Song, Guoxian Wang, Chao Shi, Yichun Chen, Zeyuan Zhou, Shijie Luo, Linjie Wetzstein, Gordon Soleymani, Mohammad |
| contents | We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating accurate expression transfer for enhanced realism in animated scenes. Together, these components form a unified framework capable of learning physical human motion and natural scene dynamics from a diverse blend of human and scene videos. Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna outperforms state-of-the-art methods, creating highly lifelike and expressive animations. The code is available at https://github.com/bytedance/X-Dyna. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_10021 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | X-Dyna: Expressive Dynamic Human Image Animation Chang, Di Xu, Hongyi Xie, You Gao, Yipeng Kuang, Zhengfei Cai, Shengqu Zhang, Chenxu Song, Guoxian Wang, Chao Shi, Yichun Chen, Zeyuan Zhou, Shijie Luo, Linjie Wetzstein, Gordon Soleymani, Mohammad Computer Vision and Pattern Recognition We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating accurate expression transfer for enhanced realism in animated scenes. Together, these components form a unified framework capable of learning physical human motion and natural scene dynamics from a diverse blend of human and scene videos. Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna outperforms state-of-the-art methods, creating highly lifelike and expressive animations. The code is available at https://github.com/bytedance/X-Dyna. |
| title | X-Dyna: Expressive Dynamic Human Image Animation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.10021 |