Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.14930 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917026308554752 |
|---|---|
| author | Huang, Binghao Xu, Jie Akinola, Iretiayo Yang, Wei Sundaralingam, Balakumar O'Flaherty, Rowland Fox, Dieter Wang, Xiaolong Mousavian, Arsalan Chao, Yu-Wei Li, Yunzhu |
| author_facet | Huang, Binghao Xu, Jie Akinola, Iretiayo Yang, Wei Sundaralingam, Balakumar O'Flaherty, Rowland Fox, Dieter Wang, Xiaolong Mousavian, Arsalan Chao, Yu-Wei Li, Yunzhu |
| contents | Humans excel at bimanual assembly tasks by adapting to rich tactile feedback -- a capability that remains difficult to replicate in robots through behavioral cloning alone, due to the suboptimality and limited diversity of human demonstrations. In this work, we present VT-Refine, a visuo-tactile policy learning framework that combines real-world demonstrations, high-fidelity tactile simulation, and reinforcement learning to tackle precise, contact-rich bimanual assembly. We begin by training a diffusion policy on a small set of demonstrations using synchronized visual and tactile inputs. This policy is then transferred to a simulated digital twin equipped with simulated tactile sensors and further refined via large-scale reinforcement learning to enhance robustness and generalization. To enable accurate sim-to-real transfer, we leverage high-resolution piezoresistive tactile sensors that provide normal force signals and can be realistically modeled in parallel using GPU-accelerated simulation. Experimental results show that VT-Refine improves assembly performance in both simulation and the real world by increasing data diversity and enabling more effective policy fine-tuning. Our project page is available at https://binghao-huang.github.io/vt_refine/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14930 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning Huang, Binghao Xu, Jie Akinola, Iretiayo Yang, Wei Sundaralingam, Balakumar O'Flaherty, Rowland Fox, Dieter Wang, Xiaolong Mousavian, Arsalan Chao, Yu-Wei Li, Yunzhu Robotics Machine Learning Humans excel at bimanual assembly tasks by adapting to rich tactile feedback -- a capability that remains difficult to replicate in robots through behavioral cloning alone, due to the suboptimality and limited diversity of human demonstrations. In this work, we present VT-Refine, a visuo-tactile policy learning framework that combines real-world demonstrations, high-fidelity tactile simulation, and reinforcement learning to tackle precise, contact-rich bimanual assembly. We begin by training a diffusion policy on a small set of demonstrations using synchronized visual and tactile inputs. This policy is then transferred to a simulated digital twin equipped with simulated tactile sensors and further refined via large-scale reinforcement learning to enhance robustness and generalization. To enable accurate sim-to-real transfer, we leverage high-resolution piezoresistive tactile sensors that provide normal force signals and can be realistically modeled in parallel using GPU-accelerated simulation. Experimental results show that VT-Refine improves assembly performance in both simulation and the real world by increasing data diversity and enabling more effective policy fine-tuning. Our project page is available at https://binghao-huang.github.io/vt_refine/. |
| title | VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuning |
| topic | Robotics Machine Learning |
| url | https://arxiv.org/abs/2510.14930 |