Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917606739410944 |
|---|---|
| author | Allshire, Arthur Mittal, Mayank Lodaya, Varun Makoviychuk, Viktor Makoviichuk, Denys Widmaier, Felix Wüthrich, Manuel Bauer, Stefan Handa, Ankur Garg, Animesh |
| author_facet | Allshire, Arthur Mittal, Mayank Lodaya, Varun Makoviychuk, Viktor Makoviichuk, Denys Widmaier, Felix Wüthrich, Manuel Bauer, Stefan Handa, Ankur Garg, Animesh |
| contents | We present a system for learning a challenging dexterous manipulation task involving moving a cube to an arbitrary 6-DoF pose with only 3-fingers trained with NVIDIA's IsaacGym simulator. We show empirical benefits, both in simulation and sim-to-real transfer, of using keypoints as opposed to position+quaternion representations for the object pose in 6-DoF for policy observations and in reward calculation to train a model-free reinforcement learning agent. By utilizing domain randomization strategies along with the keypoint representation of the pose of the manipulated object, we achieve a high success rate of 83% on a remote TriFinger system maintained by the organizers of the Real Robot Challenge. With the aim of assisting further research in learning in-hand manipulation, we make the codebase of our system, along with trained checkpoints that come with billions of steps of experience available, at https://s2r2-ig.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2108_09779 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger Allshire, Arthur Mittal, Mayank Lodaya, Varun Makoviychuk, Viktor Makoviichuk, Denys Widmaier, Felix Wüthrich, Manuel Bauer, Stefan Handa, Ankur Garg, Animesh Robotics Machine Learning We present a system for learning a challenging dexterous manipulation task involving moving a cube to an arbitrary 6-DoF pose with only 3-fingers trained with NVIDIA's IsaacGym simulator. We show empirical benefits, both in simulation and sim-to-real transfer, of using keypoints as opposed to position+quaternion representations for the object pose in 6-DoF for policy observations and in reward calculation to train a model-free reinforcement learning agent. By utilizing domain randomization strategies along with the keypoint representation of the pose of the manipulated object, we achieve a high success rate of 83% on a remote TriFinger system maintained by the organizers of the Real Robot Challenge. With the aim of assisting further research in learning in-hand manipulation, we make the codebase of our system, along with trained checkpoints that come with billions of steps of experience available, at https://s2r2-ig.github.io |
| title | Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger |
| topic | Robotics Machine Learning |
| url | https://arxiv.org/abs/2108.09779 |