Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.10881 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913077622996992 |
|---|---|
| author | Gong, Kehong Wen, Zhengyu He, Weixia Xu, Mingxi Wang, Qi Zhang, Ning Li, Zhengyu Lian, Dongze Zhao, Wei He, Xiaoyu Zhang, Mingyuan |
| author_facet | Gong, Kehong Wen, Zhengyu He, Weixia Xu, Mingxi Wang, Qi Zhang, Ning Li, Zhengyu Lian, Dongze Zhao, Wei He, Xiaoyu Zhang, Mingyuan |
| contents | Motion capture now underpins content creation far beyond digital humans, yet most existing pipelines remain species- or template-specific. We formalize this gap as Category-Agnostic Motion Capture (CAMoCap): given a monocular video and an arbitrary rigged 3D asset as a prompt, the goal is to reconstruct a rotation-based animation such as BVH that directly drives the specific asset. We present MoCapAnything, a reference-guided, factorized framework that first predicts 3D joint trajectories and then recovers asset-specific rotations via constraint-aware inverse kinematics. The system contains three learnable modules and a lightweight IK stage: (1) a Reference Prompt Encoder that extracts per-joint queries from the asset's skeleton, mesh, and rendered images; (2) a Video Feature Extractor that computes dense visual descriptors and reconstructs a coarse 4D deforming mesh to bridge the gap between video and joint space; and (3) a Unified Motion Decoder that fuses these cues to produce temporally coherent trajectories. We also curate Truebones Zoo with 1038 motion clips, each providing a standardized skeleton-mesh-render triad. Experiments on both in-domain benchmarks and in-the-wild videos show that MoCapAnything delivers high-quality skeletal animations and exhibits meaningful cross-species retargeting across heterogeneous rigs, enabling scalable, prompt-driven 3D motion capture for arbitrary assets. Project page: https://animotionlab.github.io/MoCapAnything/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_10881 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos Gong, Kehong Wen, Zhengyu He, Weixia Xu, Mingxi Wang, Qi Zhang, Ning Li, Zhengyu Lian, Dongze Zhao, Wei He, Xiaoyu Zhang, Mingyuan Computer Vision and Pattern Recognition Motion capture now underpins content creation far beyond digital humans, yet most existing pipelines remain species- or template-specific. We formalize this gap as Category-Agnostic Motion Capture (CAMoCap): given a monocular video and an arbitrary rigged 3D asset as a prompt, the goal is to reconstruct a rotation-based animation such as BVH that directly drives the specific asset. We present MoCapAnything, a reference-guided, factorized framework that first predicts 3D joint trajectories and then recovers asset-specific rotations via constraint-aware inverse kinematics. The system contains three learnable modules and a lightweight IK stage: (1) a Reference Prompt Encoder that extracts per-joint queries from the asset's skeleton, mesh, and rendered images; (2) a Video Feature Extractor that computes dense visual descriptors and reconstructs a coarse 4D deforming mesh to bridge the gap between video and joint space; and (3) a Unified Motion Decoder that fuses these cues to produce temporally coherent trajectories. We also curate Truebones Zoo with 1038 motion clips, each providing a standardized skeleton-mesh-render triad. Experiments on both in-domain benchmarks and in-the-wild videos show that MoCapAnything delivers high-quality skeletal animations and exhibits meaningful cross-species retargeting across heterogeneous rigs, enabling scalable, prompt-driven 3D motion capture for arbitrary assets. Project page: https://animotionlab.github.io/MoCapAnything/ |
| title | MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.10881 |