Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916376683216896 |
|---|---|
| author | Zhang, Zeyu Wang, Yiran Wu, Biao Chen, Shuo Zhang, Zhiyuan Huang, Shiya Zhang, Wenbo Fang, Meng Chen, Ling Zhao, Yang |
| author_facet | Zhang, Zeyu Wang, Yiran Wu, Biao Chen, Shuo Zhang, Zhiyuan Huang, Shiya Zhang, Wenbo Fang, Meng Chen, Ling Zhao, Yang |
| contents | In recent years, there has been significant interest in creating 3D avatars and motions, driven by their diverse applications in areas like film-making, video games, AR/VR, and human-robot interaction. However, current efforts primarily concentrate on either generating the 3D avatar mesh alone or producing motion sequences, with integrating these two aspects proving to be a persistent challenge. Additionally, while avatar and motion generation predominantly target humans, extending these techniques to animals remains a significant challenge due to inadequate training data and methods. To bridge these gaps, our paper presents three key contributions. Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries. The method significantly advanced the progress in dynamic 3D character generation. Secondly, we introduced a LLM planner that coordinates both motion and avatar generation, which transforms a discriminative planning into a customizable Q&A fashion. Lastly, we presented an animal motion dataset named Zoo-300K, comprising approximately 300,000 text-motion pairs across 65 animal categories and its building pipeline ZooGen, which serves as a valuable resource for the community. See project website https://steve-zeyu-zhang.github.io/MotionAvatar/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11286 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion Zhang, Zeyu Wang, Yiran Wu, Biao Chen, Shuo Zhang, Zhiyuan Huang, Shiya Zhang, Wenbo Fang, Meng Chen, Ling Zhao, Yang Computer Vision and Pattern Recognition In recent years, there has been significant interest in creating 3D avatars and motions, driven by their diverse applications in areas like film-making, video games, AR/VR, and human-robot interaction. However, current efforts primarily concentrate on either generating the 3D avatar mesh alone or producing motion sequences, with integrating these two aspects proving to be a persistent challenge. Additionally, while avatar and motion generation predominantly target humans, extending these techniques to animals remains a significant challenge due to inadequate training data and methods. To bridge these gaps, our paper presents three key contributions. Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries. The method significantly advanced the progress in dynamic 3D character generation. Secondly, we introduced a LLM planner that coordinates both motion and avatar generation, which transforms a discriminative planning into a customizable Q&A fashion. Lastly, we presented an animal motion dataset named Zoo-300K, comprising approximately 300,000 text-motion pairs across 65 animal categories and its building pipeline ZooGen, which serves as a valuable resource for the community. See project website https://steve-zeyu-zhang.github.io/MotionAvatar/ |
| title | Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.11286 |