D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917303141007360 |
|---|---|
| author | Lou, Haozhe Zhang, Mingtong Geng, Haoran Zhou, Hanyang He, Sicheng Gao, Zhiyuan Zhao, Siheng Mao, Jiageng Abbeel, Pieter Malik, Jitendra Seita, Daniel Wang, Yue |
| author_facet | Lou, Haozhe Zhang, Mingtong Geng, Haoran Zhou, Hanyang He, Sicheng Gao, Zhiyuan Zhao, Siheng Mao, Jiageng Abbeel, Pieter Malik, Jitendra Seita, Daniel Wang, Yue |
| contents | Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant challenge, especially in physical parameter identification. In this work, we introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine, enabling object mass identification from real-world visual observations and robot control signals, while enabling grasping policy learning simultaneously. Through optimizing the mass of the manipulated object, our method automatically builds high-fidelity and physically plausible digital twins. Additionally, we propose a novel approach to train force-aware grasping policies from limited data by transferring feasible human demonstrations into simulated robot demonstrations. Through comprehensive experiments, we demonstrate that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values. Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping, effectively reducing the sim-to-real gap. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01151 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping Lou, Haozhe Zhang, Mingtong Geng, Haoran Zhou, Hanyang He, Sicheng Gao, Zhiyuan Zhao, Siheng Mao, Jiageng Abbeel, Pieter Malik, Jitendra Seita, Daniel Wang, Yue Robotics Computer Vision and Pattern Recognition Graphics Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant challenge, especially in physical parameter identification. In this work, we introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine, enabling object mass identification from real-world visual observations and robot control signals, while enabling grasping policy learning simultaneously. Through optimizing the mass of the manipulated object, our method automatically builds high-fidelity and physically plausible digital twins. Additionally, we propose a novel approach to train force-aware grasping policies from limited data by transferring feasible human demonstrations into simulated robot demonstrations. Through comprehensive experiments, we demonstrate that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values. Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping, effectively reducing the sim-to-real gap. |
| title | D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping |
| topic | Robotics Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2603.01151 |