MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929280292749312 |
|---|---|
| author | Zhang, Yaqi Huang, Di Liu, Bin Tang, Shixiang Lu, Yan Chen, Lu Bai, Lei Chu, Qi Yu, Nenghai Ouyang, Wanli |
| author_facet | Zhang, Yaqi Huang, Di Liu, Bin Tang, Shixiang Lu, Yan Chen, Lu Bai, Lei Chu, Qi Yu, Nenghai Ouyang, Wanli |
| contents | Generating realistic human motion from given action descriptions has experienced significant advancements because of the emerging requirement of digital humans. While recent works have achieved impressive results in generating motion directly from textual action descriptions, they often support only a single modality of the control signal, which limits their application in the real digital human industry. This paper presents a Motion General-Purpose generaTor (MotionGPT) that can use multimodal control signals, e.g., text and single-frame poses, for generating consecutive human motions by treating multimodal signals as special input tokens in large language models (LLMs). Specifically, we first quantize multimodal control signals into discrete codes and then formulate them in a unified prompt instruction to ask the LLMs to generate the motion answer. Our MotionGPT demonstrates a unified human motion generation model with multimodal control signals by tuning a mere 0.4% of LLM parameters. To the best of our knowledge, MotionGPT is the first method to generate human motion by multimodal control signals, which we hope can shed light on this new direction. Visit our webpage at https://qiqiapink.github.io/MotionGPT/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_10900 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators Zhang, Yaqi Huang, Di Liu, Bin Tang, Shixiang Lu, Yan Chen, Lu Bai, Lei Chu, Qi Yu, Nenghai Ouyang, Wanli Computer Vision and Pattern Recognition Artificial Intelligence Generating realistic human motion from given action descriptions has experienced significant advancements because of the emerging requirement of digital humans. While recent works have achieved impressive results in generating motion directly from textual action descriptions, they often support only a single modality of the control signal, which limits their application in the real digital human industry. This paper presents a Motion General-Purpose generaTor (MotionGPT) that can use multimodal control signals, e.g., text and single-frame poses, for generating consecutive human motions by treating multimodal signals as special input tokens in large language models (LLMs). Specifically, we first quantize multimodal control signals into discrete codes and then formulate them in a unified prompt instruction to ask the LLMs to generate the motion answer. Our MotionGPT demonstrates a unified human motion generation model with multimodal control signals by tuning a mere 0.4% of LLM parameters. To the best of our knowledge, MotionGPT is the first method to generate human motion by multimodal control signals, which we hope can shed light on this new direction. Visit our webpage at https://qiqiapink.github.io/MotionGPT/. |
| title | MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2306.10900 |