HumanMM: Global Human Motion Recovery from Multi-shot Videos
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909532812214272 |
|---|---|
| author | Zhang, Yuhong Wu, Guanlin Chen, Ling-Hao Zhao, Zhuokai Lin, Jing Jiang, Xiaoke Wu, Jiamin Li, Zhuoheng Yang, Hao Frank Wang, Haoqian Zhang, Lei |
| author_facet | Zhang, Yuhong Wu, Guanlin Chen, Ling-Hao Zhao, Zhuokai Lin, Jing Jiang, Xiaoke Wu, Jiamin Li, Zhuoheng Yang, Hao Frank Wang, Haoqian Zhang, Lei |
| contents | In this paper, we present a novel framework designed to reconstruct long-sequence 3D human motion in the world coordinates from in-the-wild videos with multiple shot transitions. Such long-sequence in-the-wild motions are highly valuable to applications such as motion generation and motion understanding, but are of great challenge to be recovered due to abrupt shot transitions, partial occlusions, and dynamic backgrounds presented in such videos. Existing methods primarily focus on single-shot videos, where continuity is maintained within a single camera view, or simplify multi-shot alignment in camera space only. In this work, we tackle the challenges by integrating an enhanced camera pose estimation with Human Motion Recovery (HMR) by incorporating a shot transition detector and a robust alignment module for accurate pose and orientation continuity across shots. By leveraging a custom motion integrator, we effectively mitigate the problem of foot sliding and ensure temporal consistency in human pose. Extensive evaluations on our created multi-shot dataset from public 3D human datasets demonstrate the robustness of our method in reconstructing realistic human motion in world coordinates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07597 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HumanMM: Global Human Motion Recovery from Multi-shot Videos Zhang, Yuhong Wu, Guanlin Chen, Ling-Hao Zhao, Zhuokai Lin, Jing Jiang, Xiaoke Wu, Jiamin Li, Zhuoheng Yang, Hao Frank Wang, Haoqian Zhang, Lei Computer Vision and Pattern Recognition In this paper, we present a novel framework designed to reconstruct long-sequence 3D human motion in the world coordinates from in-the-wild videos with multiple shot transitions. Such long-sequence in-the-wild motions are highly valuable to applications such as motion generation and motion understanding, but are of great challenge to be recovered due to abrupt shot transitions, partial occlusions, and dynamic backgrounds presented in such videos. Existing methods primarily focus on single-shot videos, where continuity is maintained within a single camera view, or simplify multi-shot alignment in camera space only. In this work, we tackle the challenges by integrating an enhanced camera pose estimation with Human Motion Recovery (HMR) by incorporating a shot transition detector and a robust alignment module for accurate pose and orientation continuity across shots. By leveraging a custom motion integrator, we effectively mitigate the problem of foot sliding and ensure temporal consistency in human pose. Extensive evaluations on our created multi-shot dataset from public 3D human datasets demonstrate the robustness of our method in reconstructing realistic human motion in world coordinates. |
| title | HumanMM: Global Human Motion Recovery from Multi-shot Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.07597 |