Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.20210 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918131837960192 |
|---|---|
| author | Li, Xiaodi Xie, Pan Ren, Yi Gan, Qijun Zhang, Chen Kong, Fangyuan Yin, Xiang Peng, Bingyue Yuan, Zehuan |
| author_facet | Li, Xiaodi Xie, Pan Ren, Yi Gan, Qijun Zhang, Chen Kong, Fangyuan Yin, Xiang Peng, Bingyue Yuan, Zehuan |
| contents | Audio-driven human animation has attracted wide attention thanks to its practical applications. However, critical challenges remain in generating high-resolution, long-duration videos with consistent appearance and natural hand motions. Existing methods extend videos using overlapping motion frames but suffer from error accumulation, leading to identity drift, color shifts, and scene instability. Additionally, hand movements are poorly modeled, resulting in noticeable distortions and misalignment with the audio. In this work, we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner. Since pose sequences are decoupled from appearance and resist temporal degradation, our pose-guided refiner employs stable poses and the initial frame as a visual anchor to reduce drift and improve lip synchronization. Moreover, to enhance semantic accuracy and gesture realism, we introduce a hand-specific reward mechanism trained with high-quality hand motion data. Experiments on the EMTD and HDTF datasets show that InfinityHuman achieves state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync. Ablation studies further confirm the effectiveness of each module. Code will be made public. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20210 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | InfinityHuman: Towards Long-Term Audio-Driven Human Li, Xiaodi Xie, Pan Ren, Yi Gan, Qijun Zhang, Chen Kong, Fangyuan Yin, Xiang Peng, Bingyue Yuan, Zehuan Computer Vision and Pattern Recognition Audio-driven human animation has attracted wide attention thanks to its practical applications. However, critical challenges remain in generating high-resolution, long-duration videos with consistent appearance and natural hand motions. Existing methods extend videos using overlapping motion frames but suffer from error accumulation, leading to identity drift, color shifts, and scene instability. Additionally, hand movements are poorly modeled, resulting in noticeable distortions and misalignment with the audio. In this work, we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner. Since pose sequences are decoupled from appearance and resist temporal degradation, our pose-guided refiner employs stable poses and the initial frame as a visual anchor to reduce drift and improve lip synchronization. Moreover, to enhance semantic accuracy and gesture realism, we introduce a hand-specific reward mechanism trained with high-quality hand motion data. Experiments on the EMTD and HDTF datasets show that InfinityHuman achieves state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync. Ablation studies further confirm the effectiveness of each module. Code will be made public. |
| title | InfinityHuman: Towards Long-Term Audio-Driven Human |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.20210 |