Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.17610 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911819970379776 |
|---|---|
| author | Zhang, He Ren, Shenghao Yuan, Haolei Zhao, Jianhui Li, Fan Sun, Shuangpeng Liang, Zhenghao Yu, Tao Shen, Qiu Cao, Xun |
| author_facet | Zhang, He Ren, Shenghao Yuan, Haolei Zhao, Jianhui Li, Fan Sun, Shuangpeng Liang, Zhenghao Yu, Tao Shen, Qiu Cao, Xun |
| contents | Foot contact is an important cue for human motion capture, understanding, and generation. Existing datasets tend to annotate dense foot contact using visual matching with thresholding or incorporating pressure signals. However, these approaches either suffer from low accuracy or are only designed for small-range and slow motion. There is still a lack of a vision-pressure multimodal dataset with large-range and fast human motion, as well as accurate and dense foot-contact annotation. To fill this gap, we propose a Multimodal MoCap Dataset with Vision and Pressure sensors, named MMVP. MMVP provides accurate and dense plantar pressure signals synchronized with RGBD observations, which is especially useful for both plausible shape estimation, robust pose fitting without foot drifting, and accurate global translation tracking. To validate the dataset, we propose an RGBD-P SMPL fitting method and also a monocular-video-based baseline framework, VP-MoCap, for human motion capture. Experiments demonstrate that our RGBD-P SMPL Fitting results significantly outperform pure visual motion capture. Moreover, VP-MoCap outperforms SOTA methods in foot-contact and global translation estimation accuracy. We believe the configuration of the dataset and the baseline frameworks will stimulate the research in this direction and also provide a good reference for MoCap applications in various domains. Project page: https://metaverse-ai-lab-thu.github.io/MMVP-Dataset/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_17610 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors Zhang, He Ren, Shenghao Yuan, Haolei Zhao, Jianhui Li, Fan Sun, Shuangpeng Liang, Zhenghao Yu, Tao Shen, Qiu Cao, Xun Computer Vision and Pattern Recognition Foot contact is an important cue for human motion capture, understanding, and generation. Existing datasets tend to annotate dense foot contact using visual matching with thresholding or incorporating pressure signals. However, these approaches either suffer from low accuracy or are only designed for small-range and slow motion. There is still a lack of a vision-pressure multimodal dataset with large-range and fast human motion, as well as accurate and dense foot-contact annotation. To fill this gap, we propose a Multimodal MoCap Dataset with Vision and Pressure sensors, named MMVP. MMVP provides accurate and dense plantar pressure signals synchronized with RGBD observations, which is especially useful for both plausible shape estimation, robust pose fitting without foot drifting, and accurate global translation tracking. To validate the dataset, we propose an RGBD-P SMPL fitting method and also a monocular-video-based baseline framework, VP-MoCap, for human motion capture. Experiments demonstrate that our RGBD-P SMPL Fitting results significantly outperform pure visual motion capture. Moreover, VP-MoCap outperforms SOTA methods in foot-contact and global translation estimation accuracy. We believe the configuration of the dataset and the baseline frameworks will stimulate the research in this direction and also provide a good reference for MoCap applications in various domains. Project page: https://metaverse-ai-lab-thu.github.io/MMVP-Dataset/. |
| title | MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.17610 |