DexVLG: Dexterous Vision-Language-Grasp Model at Scale
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913924529520640 |
|---|---|
| author | He, Jiawei Li, Danshi Yu, Xinqiang Qi, Zekun Zhang, Wenyao Chen, Jiayi Zhang, Zhaoxiang Zhang, Zhizheng Yi, Li Wang, He |
| author_facet | He, Jiawei Li, Danshi Yu, Xinqiang Qi, Zekun Zhang, Wenyao Chen, Jiayi Zhang, Zhaoxiang Zhang, Zhizheng Yi, Li Wang, He |
| contents | As large models gain traction, vision-language-action (VLA) systems are enabling robots to tackle increasingly complex tasks. However, limited by the difficulty of data collection, progress has mainly focused on controlling simple gripper end-effectors. There is little research on functional grasping with large models for human-like dexterous hands. In this paper, we introduce DexVLG, a large Vision-Language-Grasp model for Dexterous grasp pose prediction aligned with language instructions using single-view RGBD input. To accomplish this, we generate a dataset of 170 million dexterous grasp poses mapped to semantic parts across 174,000 objects in simulation, paired with detailed part-level captions. This large-scale dataset, named DexGraspNet 3.0, is used to train a VLM and flow-matching-based pose head capable of producing instruction-aligned grasp poses for tabletop objects. To assess DexVLG's performance, we create benchmarks in physics-based simulations and conduct real-world experiments. Extensive testing demonstrates DexVLG's strong zero-shot generalization capabilities-achieving over 76% zero-shot execution success rate and state-of-the-art part-grasp accuracy in simulation-and successful part-aligned grasps on physical objects in real-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02747 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DexVLG: Dexterous Vision-Language-Grasp Model at Scale He, Jiawei Li, Danshi Yu, Xinqiang Qi, Zekun Zhang, Wenyao Chen, Jiayi Zhang, Zhaoxiang Zhang, Zhizheng Yi, Li Wang, He Computer Vision and Pattern Recognition Robotics As large models gain traction, vision-language-action (VLA) systems are enabling robots to tackle increasingly complex tasks. However, limited by the difficulty of data collection, progress has mainly focused on controlling simple gripper end-effectors. There is little research on functional grasping with large models for human-like dexterous hands. In this paper, we introduce DexVLG, a large Vision-Language-Grasp model for Dexterous grasp pose prediction aligned with language instructions using single-view RGBD input. To accomplish this, we generate a dataset of 170 million dexterous grasp poses mapped to semantic parts across 174,000 objects in simulation, paired with detailed part-level captions. This large-scale dataset, named DexGraspNet 3.0, is used to train a VLM and flow-matching-based pose head capable of producing instruction-aligned grasp poses for tabletop objects. To assess DexVLG's performance, we create benchmarks in physics-based simulations and conduct real-world experiments. Extensive testing demonstrates DexVLG's strong zero-shot generalization capabilities-achieving over 76% zero-shot execution success rate and state-of-the-art part-grasp accuracy in simulation-and successful part-aligned grasps on physical objects in real-world scenarios. |
| title | DexVLG: Dexterous Vision-Language-Grasp Model at Scale |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2507.02747 |