Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.02772 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916507875803136 |
|---|---|
| author | Shi, Xu Yao, Wei Luo, Chuanchen Peng, Junran Zhang, Hongwen Sun, Yunlian |
| author_facet | Shi, Xu Yao, Wei Luo, Chuanchen Peng, Junran Zhang, Hongwen Sun, Yunlian |
| contents | Recently, significant progress has been made in text-based motion generation, enabling the generation of diverse and high-quality human motions that conform to textual descriptions. However, generating motions beyond the distribution of original datasets remains challenging, i.e., zero-shot generation. By adopting a divide-and-conquer strategy, we propose a new framework named Fine-Grained Human Motion Diffusion Model (FG-MDM) for zero-shot human motion generation. Specifically, we first parse previous vague textual annotations into fine-grained descriptions of different body parts by leveraging a large language model. We then use these fine-grained descriptions to guide a transformer-based diffusion model, which further adopts a design of part tokens. FG-MDM can generate human motions beyond the scope of original datasets owing to descriptions that are closer to motion essence. Our experimental results demonstrate the superiority of FG-MDM over previous methods in zero-shot settings. We will release our fine-grained textual annotations for HumanML3D and KIT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_02772 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | FG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptions Shi, Xu Yao, Wei Luo, Chuanchen Peng, Junran Zhang, Hongwen Sun, Yunlian Computer Vision and Pattern Recognition Recently, significant progress has been made in text-based motion generation, enabling the generation of diverse and high-quality human motions that conform to textual descriptions. However, generating motions beyond the distribution of original datasets remains challenging, i.e., zero-shot generation. By adopting a divide-and-conquer strategy, we propose a new framework named Fine-Grained Human Motion Diffusion Model (FG-MDM) for zero-shot human motion generation. Specifically, we first parse previous vague textual annotations into fine-grained descriptions of different body parts by leveraging a large language model. We then use these fine-grained descriptions to guide a transformer-based diffusion model, which further adopts a design of part tokens. FG-MDM can generate human motions beyond the scope of original datasets owing to descriptions that are closer to motion essence. Our experimental results demonstrate the superiority of FG-MDM over previous methods in zero-shot settings. We will release our fine-grained textual annotations for HumanML3D and KIT. |
| title | FG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.02772 |