ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908932157472768 |
|---|---|
| author | Zhou, Xukun Li, Fengxin Chen, Ming Zhou, Yan Wan, Pengfei Zhang, Di Jin, Yeying Fan, Zhaoxin Liu, Hongyan He, Jun |
| author_facet | Zhou, Xukun Li, Fengxin Chen, Ming Zhou, Yan Wan, Pengfei Zhang, Di Jin, Yeying Fan, Zhaoxin Liu, Hongyan He, Jun |
| contents | Audio-driven human gesture synthesis is a crucial task with broad applications in virtual avatars, human-computer interaction, and creative content generation. Despite notable progress, existing methods often produce gestures that are coarse, lack expressiveness, and fail to fully align with audio semantics. To address these challenges, we propose ExGes, a novel retrieval-enhanced diffusion framework with three key designs: (1) a Motion Base Construction, which builds a gesture library using training dataset; (2) a Motion Retrieval Module, employing constrative learning and momentum distillation for fine-grained reference poses retreiving; and (3) a Precision Control Module, integrating partial masking and stochastic masking to enable flexible and fine-grained control. Experimental evaluations on BEAT2 demonstrate that ExGes reduces Fréchet Gesture Distance by 6.2\% and improves motion diversity by 5.3\% over EMAGE, with user studies revealing a 71.3\% preference for its naturalness and semantic relevance. Code will be released upon acceptance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06499 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis Zhou, Xukun Li, Fengxin Chen, Ming Zhou, Yan Wan, Pengfei Zhang, Di Jin, Yeying Fan, Zhaoxin Liu, Hongyan He, Jun Computer Vision and Pattern Recognition Artificial Intelligence Audio-driven human gesture synthesis is a crucial task with broad applications in virtual avatars, human-computer interaction, and creative content generation. Despite notable progress, existing methods often produce gestures that are coarse, lack expressiveness, and fail to fully align with audio semantics. To address these challenges, we propose ExGes, a novel retrieval-enhanced diffusion framework with three key designs: (1) a Motion Base Construction, which builds a gesture library using training dataset; (2) a Motion Retrieval Module, employing constrative learning and momentum distillation for fine-grained reference poses retreiving; and (3) a Precision Control Module, integrating partial masking and stochastic masking to enable flexible and fine-grained control. Experimental evaluations on BEAT2 demonstrate that ExGes reduces Fréchet Gesture Distance by 6.2\% and improves motion diversity by 5.3\% over EMAGE, with user studies revealing a 71.3\% preference for its naturalness and semantic relevance. Code will be released upon acceptance. |
| title | ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2503.06499 |