UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910066251137024 |
|---|---|
| author | Zhang, Gu Xu, Qicheng Zhang, Haozhe Ma, Jianhan He, Long Bao, Yiming Ping, Zeyu Yuan, Zhecheng Lu, Chenhao Yuan, Chengbo Liang, Tianhai Tian, Xiaoyu Shao, Maanping Zhang, Feihong Ding, Mingyu Gao, Yang Zhao, Hao Zhao, Hang Xu, Huazhe |
| author_facet | Zhang, Gu Xu, Qicheng Zhang, Haozhe Ma, Jianhan He, Long Bao, Yiming Ping, Zeyu Yuan, Zhecheng Lu, Chenhao Yuan, Chengbo Liang, Tianhai Tian, Xiaoyu Shao, Maanping Zhang, Feihong Ding, Mingyu Gao, Yang Zhao, Hao Zhao, Hang Xu, Huazhe |
| contents | Dexterous manipulation remains challenging due to the cost of collecting real-robot teleoperation data, the heterogeneity of hand embodiments, and the high dimensionality of control. We present UniDex, a robot foundation suite that couples a large-scale robot-centric dataset with a unified vision-language-action (VLA) policy and a practical human-data capture setup for universal dexterous hand control. First, we construct UniDex-Dataset, a robot-centric dataset over 50K trajectories across eight dexterous hands (6--24 DoFs), derived from egocentric human video datasets. To transform human data into robot-executable trajectories, we employ a human-in-the-loop retargeting procedure to align fingertip trajectories while preserving plausible hand-object contacts, and we operate on explicit 3D pointclouds with human hands masked to narrow kinematic and visual gaps. Second, we introduce the Function-Actuator-Aligned Space (FAAS), a unified action space that maps functionally similar actuators to shared coordinates, enabling cross-hand transfer. Leveraging FAAS as the action parameterization, we train UniDex-VLA, a 3D VLA policy pretrained on UniDex-Dataset and finetuned with task demonstrations. In addition, we build UniDex-Cap, a simple portable capture setup that records synchronized RGB-D streams and human hand poses and converts them into robot-executable trajectories to enable human-robot data co-training that reduces reliance on costly robot demonstrations. On challenging tool-use tasks across two different hands, UniDex-VLA achieves 81% average task progress and outperforms prior VLA baselines by a large margin, while exhibiting strong spatial, object, and zero-shot cross-hand generalization. Together, UniDex-Dataset, UniDex-VLA, and UniDex-Cap provide a scalable foundation suite for universal dexterous manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_22264 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos Zhang, Gu Xu, Qicheng Zhang, Haozhe Ma, Jianhan He, Long Bao, Yiming Ping, Zeyu Yuan, Zhecheng Lu, Chenhao Yuan, Chengbo Liang, Tianhai Tian, Xiaoyu Shao, Maanping Zhang, Feihong Ding, Mingyu Gao, Yang Zhao, Hao Zhao, Hang Xu, Huazhe Robotics Dexterous manipulation remains challenging due to the cost of collecting real-robot teleoperation data, the heterogeneity of hand embodiments, and the high dimensionality of control. We present UniDex, a robot foundation suite that couples a large-scale robot-centric dataset with a unified vision-language-action (VLA) policy and a practical human-data capture setup for universal dexterous hand control. First, we construct UniDex-Dataset, a robot-centric dataset over 50K trajectories across eight dexterous hands (6--24 DoFs), derived from egocentric human video datasets. To transform human data into robot-executable trajectories, we employ a human-in-the-loop retargeting procedure to align fingertip trajectories while preserving plausible hand-object contacts, and we operate on explicit 3D pointclouds with human hands masked to narrow kinematic and visual gaps. Second, we introduce the Function-Actuator-Aligned Space (FAAS), a unified action space that maps functionally similar actuators to shared coordinates, enabling cross-hand transfer. Leveraging FAAS as the action parameterization, we train UniDex-VLA, a 3D VLA policy pretrained on UniDex-Dataset and finetuned with task demonstrations. In addition, we build UniDex-Cap, a simple portable capture setup that records synchronized RGB-D streams and human hand poses and converts them into robot-executable trajectories to enable human-robot data co-training that reduces reliance on costly robot demonstrations. On challenging tool-use tasks across two different hands, UniDex-VLA achieves 81% average task progress and outperforms prior VLA baselines by a large margin, while exhibiting strong spatial, object, and zero-shot cross-hand generalization. Together, UniDex-Dataset, UniDex-VLA, and UniDex-Cap provide a scalable foundation suite for universal dexterous manipulation. |
| title | UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos |
| topic | Robotics |
| url | https://arxiv.org/abs/2603.22264 |