Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.28766 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908921965314048 |
|---|---|
| author | Zhang, Zimu Zhang, Yucheng Xu, Xiyan Wang, Ziyin Xu, Sirui Zhou, Kai Zhou, Bing Guo, Chuan Wang, Jian Wang, Yu-Xiong Gui, Liang-Yan |
| author_facet | Zhang, Zimu Zhang, Yucheng Xu, Xiyan Wang, Ziyin Xu, Sirui Zhou, Kai Zhou, Bing Guo, Chuan Wang, Jian Wang, Yu-Xiong Gui, Liang-Yan |
| contents | Synthesizing human motion has advanced rapidly, yet realistic hand motion and bimanual interaction remain underexplored. Whole-body models often miss the fine-grained cues that drive dexterous behavior, finger articulation, contact timing, and inter-hand coordination, and existing resources lack high-fidelity bimanual sequences that capture nuanced finger dynamics and collaboration. To fill this gap, we present HandX, a unified foundation spanning data, annotation, and evaluation. We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics. For scalable annotation, we introduce a decoupled strategy that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features. Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes. Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics. We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion. Our dataset is released to support future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_28766 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HandX: Scaling Bimanual Motion and Interaction Generation Zhang, Zimu Zhang, Yucheng Xu, Xiyan Wang, Ziyin Xu, Sirui Zhou, Kai Zhou, Bing Guo, Chuan Wang, Jian Wang, Yu-Xiong Gui, Liang-Yan Computer Vision and Pattern Recognition Synthesizing human motion has advanced rapidly, yet realistic hand motion and bimanual interaction remain underexplored. Whole-body models often miss the fine-grained cues that drive dexterous behavior, finger articulation, contact timing, and inter-hand coordination, and existing resources lack high-fidelity bimanual sequences that capture nuanced finger dynamics and collaboration. To fill this gap, we present HandX, a unified foundation spanning data, annotation, and evaluation. We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics. For scalable annotation, we introduce a decoupled strategy that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features. Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes. Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics. We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion. Our dataset is released to support future research. |
| title | HandX: Scaling Bimanual Motion and Interaction Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.28766 |