OmniControl: Control Any Joint at Any Time for Human Motion Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929312921288704 |
|---|---|
| author | Xie, Yiming Jampani, Varun Zhong, Lei Sun, Deqing Jiang, Huaizu |
| author_facet | Xie, Yiming Jampani, Varun Zhong, Lei Sun, Deqing Jiang, Huaizu |
| contents | We present a novel approach named OmniControl for incorporating flexible spatial control signals into a text-conditioned human motion generation model based on the diffusion process. Unlike previous methods that can only control the pelvis trajectory, OmniControl can incorporate flexible spatial control signals over different joints at different times with only one model. Specifically, we propose analytic spatial guidance that ensures the generated motion can tightly conform to the input control signals. At the same time, realism guidance is introduced to refine all the joints to generate more coherent motion. Both the spatial and realism guidance are essential and they are highly complementary for balancing control accuracy and motion realism. By combining them, OmniControl generates motions that are realistic, coherent, and consistent with the spatial constraints. Experiments on HumanML3D and KIT-ML datasets show that OmniControl not only achieves significant improvement over state-of-the-art methods on pelvis control but also shows promising results when incorporating the constraints over other joints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_08580 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | OmniControl: Control Any Joint at Any Time for Human Motion Generation Xie, Yiming Jampani, Varun Zhong, Lei Sun, Deqing Jiang, Huaizu Computer Vision and Pattern Recognition Graphics We present a novel approach named OmniControl for incorporating flexible spatial control signals into a text-conditioned human motion generation model based on the diffusion process. Unlike previous methods that can only control the pelvis trajectory, OmniControl can incorporate flexible spatial control signals over different joints at different times with only one model. Specifically, we propose analytic spatial guidance that ensures the generated motion can tightly conform to the input control signals. At the same time, realism guidance is introduced to refine all the joints to generate more coherent motion. Both the spatial and realism guidance are essential and they are highly complementary for balancing control accuracy and motion realism. By combining them, OmniControl generates motions that are realistic, coherent, and consistent with the spatial constraints. Experiments on HumanML3D and KIT-ML datasets show that OmniControl not only achieves significant improvement over state-of-the-art methods on pelvis control but also shows promising results when incorporating the constraints over other joints. |
| title | OmniControl: Control Any Joint at Any Time for Human Motion Generation |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2310.08580 |