GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908536523456512 |
|---|---|
| author | Deshpande, Abhay Deng, Yuquan Ray, Arijit Salvador, Jordi Han, Winson Duan, Jiafei Zeng, Kuo-Hao Zhu, Yuke Krishna, Ranjay Hendrix, Rose |
| author_facet | Deshpande, Abhay Deng, Yuquan Ray, Arijit Salvador, Jordi Han, Winson Duan, Jiafei Zeng, Kuo-Hao Zhu, Yuke Krishna, Ranjay Hendrix, Rose |
| contents | We present GrasMolmo, a generalizable open-vocabulary task-oriented grasping (TOG) model. GraspMolmo predicts semantically appropriate, stable grasps conditioned on a natural language instruction and a single RGB-D frame. For instance, given "pour me some tea", GraspMolmo selects a grasp on a teapot handle rather than its body. Unlike prior TOG methods, which are limited by small datasets, simplistic language, and uncluttered scenes, GraspMolmo learns from PRISM, a novel large-scale synthetic dataset of 379k samples featuring cluttered environments and diverse, realistic task descriptions. We fine-tune the Molmo visual-language model on this data, enabling GraspMolmo to generalize to novel open-vocabulary instructions and objects. In challenging real-world evaluations, GraspMolmo achieves state-of-the-art results, with a 70% prediction success on complex tasks, compared to the 35% achieved by the next best alternative. GraspMolmo also successfully demonstrates the ability to predict semantically correct bimanual grasps zero-shot. We release our synthetic dataset, code, model, and benchmarks to accelerate research in task-semantic robotic manipulation, which, along with videos, are available at https://abhaybd.github.io/GraspMolmo/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13441 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation Deshpande, Abhay Deng, Yuquan Ray, Arijit Salvador, Jordi Han, Winson Duan, Jiafei Zeng, Kuo-Hao Zhu, Yuke Krishna, Ranjay Hendrix, Rose Robotics We present GrasMolmo, a generalizable open-vocabulary task-oriented grasping (TOG) model. GraspMolmo predicts semantically appropriate, stable grasps conditioned on a natural language instruction and a single RGB-D frame. For instance, given "pour me some tea", GraspMolmo selects a grasp on a teapot handle rather than its body. Unlike prior TOG methods, which are limited by small datasets, simplistic language, and uncluttered scenes, GraspMolmo learns from PRISM, a novel large-scale synthetic dataset of 379k samples featuring cluttered environments and diverse, realistic task descriptions. We fine-tune the Molmo visual-language model on this data, enabling GraspMolmo to generalize to novel open-vocabulary instructions and objects. In challenging real-world evaluations, GraspMolmo achieves state-of-the-art results, with a 70% prediction success on complex tasks, compared to the 35% achieved by the next best alternative. GraspMolmo also successfully demonstrates the ability to predict semantically correct bimanual grasps zero-shot. We release our synthetic dataset, code, model, and benchmarks to accelerate research in task-semantic robotic manipulation, which, along with videos, are available at https://abhaybd.github.io/GraspMolmo/. |
| title | GraspMolmo: Generalizable Task-Oriented Grasping via Large-Scale Synthetic Data Generation |
| topic | Robotics |
| url | https://arxiv.org/abs/2505.13441 |