Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.19580 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909328950165504 |
|---|---|
| author | Xu, Zhongcong Song, Chaoyue Song, Guoxian Zhang, Jianfeng Liew, Jun Hao Xu, Hongyi Xie, You Luo, Linjie Lin, Guosheng Feng, Jiashi Shou, Mike Zheng |
| author_facet | Xu, Zhongcong Song, Chaoyue Song, Guoxian Zhang, Jianfeng Liew, Jun Hao Xu, Hongyi Xie, You Luo, Linjie Lin, Guosheng Feng, Jiashi Shou, Mike Zheng |
| contents | Recent advances in video diffusion models have enabled realistic and controllable human image animation with temporal coherence. Although generating reasonable results, existing methods often overlook the need for regional supervision in crucial areas such as the face and hands, and neglect the explicit modeling for motion blur, leading to unrealistic low-quality synthesis. To address these limitations, we first leverage regional supervision for detailed regions to enhance face and hand faithfulness. Second, we model the motion blur explicitly to further improve the appearance quality. Third, we explore novel training strategies for high-resolution human animation to improve the overall fidelity. Experimental results demonstrate that our proposed method outperforms state-of-the-art approaches, achieving significant improvements upon the strongest baseline by more than 21.0% and 57.4% in terms of reconstruction precision (L1) and perceptual quality (FVD) on HumanDance dataset. Code and model will be made available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19580 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | High Quality Human Image Animation using Regional Supervision and Motion Blur Condition Xu, Zhongcong Song, Chaoyue Song, Guoxian Zhang, Jianfeng Liew, Jun Hao Xu, Hongyi Xie, You Luo, Linjie Lin, Guosheng Feng, Jiashi Shou, Mike Zheng Computer Vision and Pattern Recognition Recent advances in video diffusion models have enabled realistic and controllable human image animation with temporal coherence. Although generating reasonable results, existing methods often overlook the need for regional supervision in crucial areas such as the face and hands, and neglect the explicit modeling for motion blur, leading to unrealistic low-quality synthesis. To address these limitations, we first leverage regional supervision for detailed regions to enhance face and hand faithfulness. Second, we model the motion blur explicitly to further improve the appearance quality. Third, we explore novel training strategies for high-resolution human animation to improve the overall fidelity. Experimental results demonstrate that our proposed method outperforms state-of-the-art approaches, achieving significant improvements upon the strongest baseline by more than 21.0% and 57.4% in terms of reconstruction precision (L1) and perceptual quality (FVD) on HumanDance dataset. Code and model will be made available. |
| title | High Quality Human Image Animation using Regional Supervision and Motion Blur Condition |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.19580 |