Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911727491219456 |
|---|---|
| author | Xie, Liang Yu, Hongxiang Xu, Kechun Yang, Tong Wang, Minhang Lu, Haojian Xiong, Rong Wang, Yue |
| author_facet | Xie, Liang Yu, Hongxiang Xu, Kechun Yang, Tong Wang, Minhang Lu, Haojian Xiong, Rong Wang, Yue |
| contents | This paper proposes a learning-based visual peg-in-hole that enables training with several shapes in simulation, and adapting to arbitrary unseen shapes in real world with minimal sim-to-real cost. The core idea is to decouple the generalization of the sensory-motor policy to the design of a fast-adaptable perception module and a simulated generic policy module. The framework consists of a segmentation network (SN), a virtual sensor network (VSN), and a controller network (CN). Concretely, the VSN is trained to measure the pose of the unseen shape from a segmented image. After that, given the shape-agnostic pose measurement, the CN is trained to achieve generic peg-in-hole. Finally, when applying to real unseen holes, we only have to fine-tune the SN required by the simulated VSN+CN. To further minimize the transfer cost, we propose to automatically collect and annotate the data for the SN after one-minute human teaching. Simulated and real-world results are presented under the configurations of eye-to/in-hand. An electric vehicle charging system with the proposed policy inside achieves a 10/10 success rate in 2-3s, using only hundreds of auto-labeled samples for the SN transfer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2205_04297 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes Xie, Liang Yu, Hongxiang Xu, Kechun Yang, Tong Wang, Minhang Lu, Haojian Xiong, Rong Wang, Yue Robotics Artificial Intelligence This paper proposes a learning-based visual peg-in-hole that enables training with several shapes in simulation, and adapting to arbitrary unseen shapes in real world with minimal sim-to-real cost. The core idea is to decouple the generalization of the sensory-motor policy to the design of a fast-adaptable perception module and a simulated generic policy module. The framework consists of a segmentation network (SN), a virtual sensor network (VSN), and a controller network (CN). Concretely, the VSN is trained to measure the pose of the unseen shape from a segmented image. After that, given the shape-agnostic pose measurement, the CN is trained to achieve generic peg-in-hole. Finally, when applying to real unseen holes, we only have to fine-tune the SN required by the simulated VSN+CN. To further minimize the transfer cost, we propose to automatically collect and annotate the data for the SN after one-minute human teaching. Simulated and real-world results are presented under the configurations of eye-to/in-hand. An electric vehicle charging system with the proposed policy inside achieves a 10/10 success rate in 2-3s, using only hundreds of auto-labeled samples for the SN transfer. |
| title | Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2205.04297 |