Ges-QA: A Multidimensional Quality Assessment Dataset for Audio-to-3D Gesture Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915891363446784 |
|---|---|
| author | Gao, Zhilin Li, Yunhao Wu, Sijing Cao, Yuqin Duan, Huiyu Zhai, Guangtao |
| author_facet | Gao, Zhilin Li, Yunhao Wu, Sijing Cao, Yuqin Duan, Huiyu Zhai, Guangtao |
| contents | The Audio-to-3D-Gesture (A2G) task has enormous potential for various applications in virtual reality and computer graphics, etc. However, current evaluation metrics, such as Fréchet Gesture Distance or Beat Constancy, fail at reflecting the human preference of the generated 3D gestures. To cope with this problem, exploring human preference and an objective quality assessment metric for AI-generated 3D human gestures is becoming increasingly significant. In this paper, we introduce the Ges-QA dataset, which includes 1,400 samples with multidimensional scores for gesture quality and audio-gesture consistency. Moreover, we collect binary classification labels to determine whether the generated gestures match the emotions of the audio. Equipped with our Ges-QA dataset, we propose a multi-modal transformer-based neural network with 3 branches for video, audio and 3D skeleton modalities, which can score A2G contents in multiple dimensions. Comparative experimental results and ablation studies demonstrate that Ges-QAer yields state-of-the-art performance on our dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_12020 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Ges-QA: A Multidimensional Quality Assessment Dataset for Audio-to-3D Gesture Generation Gao, Zhilin Li, Yunhao Wu, Sijing Cao, Yuqin Duan, Huiyu Zhai, Guangtao Multimedia The Audio-to-3D-Gesture (A2G) task has enormous potential for various applications in virtual reality and computer graphics, etc. However, current evaluation metrics, such as Fréchet Gesture Distance or Beat Constancy, fail at reflecting the human preference of the generated 3D gestures. To cope with this problem, exploring human preference and an objective quality assessment metric for AI-generated 3D human gestures is becoming increasingly significant. In this paper, we introduce the Ges-QA dataset, which includes 1,400 samples with multidimensional scores for gesture quality and audio-gesture consistency. Moreover, we collect binary classification labels to determine whether the generated gestures match the emotions of the audio. Equipped with our Ges-QA dataset, we propose a multi-modal transformer-based neural network with 3 branches for video, audio and 3D skeleton modalities, which can score A2G contents in multiple dimensions. Comparative experimental results and ablation studies demonstrate that Ges-QAer yields state-of-the-art performance on our dataset. |
| title | Ges-QA: A Multidimensional Quality Assessment Dataset for Audio-to-3D Gesture Generation |
| topic | Multimedia |
| url | https://arxiv.org/abs/2508.12020 |