Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.16498 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915033448972288 |
|---|---|
| author | Moreno-Villamarín, David Eduardo Hilsmann, Anna Eisert, Peter |
| author_facet | Moreno-Villamarín, David Eduardo Hilsmann, Anna Eisert, Peter |
| contents | We present a generative model that learns to synthesize human motion from limited training sequences. Our framework provides conditional generation and blending across multiple temporal resolutions. The model adeptly captures human motion patterns by integrating skeletal convolution layers and a multi-scale architecture. Our model contains a set of generative and adversarial networks, along with embedding modules, each tailored for generating motions at specific frame rates while exerting control over their content and details. Notably, our approach also extends to the synthesis of co-speech gestures, demonstrating its ability to generate synchronized gestures from speech inputs, even with limited paired data. Through direct synthesis of SMPL pose parameters, our approach avoids test-time adjustments to fit human body meshes. Experimental results showcase our model's ability to achieve extensive coverage of training examples, while generating diverse motions, as indicated by local and global diversity metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_16498 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-Resolution Generative Modeling of Human Motion from Limited Data Moreno-Villamarín, David Eduardo Hilsmann, Anna Eisert, Peter Computer Vision and Pattern Recognition Graphics Machine Learning I.3 We present a generative model that learns to synthesize human motion from limited training sequences. Our framework provides conditional generation and blending across multiple temporal resolutions. The model adeptly captures human motion patterns by integrating skeletal convolution layers and a multi-scale architecture. Our model contains a set of generative and adversarial networks, along with embedding modules, each tailored for generating motions at specific frame rates while exerting control over their content and details. Notably, our approach also extends to the synthesis of co-speech gestures, demonstrating its ability to generate synchronized gestures from speech inputs, even with limited paired data. Through direct synthesis of SMPL pose parameters, our approach avoids test-time adjustments to fit human body meshes. Experimental results showcase our model's ability to achieve extensive coverage of training examples, while generating diverse motions, as indicated by local and global diversity metrics. |
| title | Multi-Resolution Generative Modeling of Human Motion from Limited Data |
| topic | Computer Vision and Pattern Recognition Graphics Machine Learning I.3 |
| url | https://arxiv.org/abs/2411.16498 |