Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.21616 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913762703835136 |
|---|---|
| author | Chen, Jiahui Huan, Yang Shi, Runhua Ding, Chanfan Mo, Xiaoqi Xiong, Siyu He, Yinong |
| author_facet | Chen, Jiahui Huan, Yang Shi, Runhua Ding, Chanfan Mo, Xiaoqi Xiong, Siyu He, Yinong |
| contents | Gestures are essential for enhancing co-speech communication, offering visual emphasis and complementing verbal interactions. While prior work has concentrated on point-level motion or fully supervised data-driven methods, we focus on co-speech gestures, advocating for weakly supervised learning and pixel-level motion deviations. We introduce a weakly supervised framework that learns latent representation deviations, tailored for co-speech gesture video generation. Our approach employs a diffusion model to integrate latent motion features, enabling more precise and nuanced gesture representation. By leveraging weakly supervised deviations in latent space, we effectively generate hand gestures and mouth movements, crucial for realistic video production. Experiments show our method significantly improves video quality, surpassing current state-of-the-art techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21616 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Audio-driven Gesture Generation via Deviation Feature in the Latent Space Chen, Jiahui Huan, Yang Shi, Runhua Ding, Chanfan Mo, Xiaoqi Xiong, Siyu He, Yinong Computer Vision and Pattern Recognition Gestures are essential for enhancing co-speech communication, offering visual emphasis and complementing verbal interactions. While prior work has concentrated on point-level motion or fully supervised data-driven methods, we focus on co-speech gestures, advocating for weakly supervised learning and pixel-level motion deviations. We introduce a weakly supervised framework that learns latent representation deviations, tailored for co-speech gesture video generation. Our approach employs a diffusion model to integrate latent motion features, enabling more precise and nuanced gesture representation. By leveraging weakly supervised deviations in latent space, we effectively generate hand gestures and mouth movements, crucial for realistic video production. Experiments show our method significantly improves video quality, surpassing current state-of-the-art techniques. |
| title | Audio-driven Gesture Generation via Deviation Feature in the Latent Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.21616 |