FBI: Learning Dexterous In-hand Manipulation with Dynamic Visuotactile Shortcut Policy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913998522286080 |
|---|---|
| author | Chen, Yijin Xu, Wenqiang Yu, Zhenjun Tang, Tutian Li, Yutong Yao, Siqiong Lu, Cewu |
| author_facet | Chen, Yijin Xu, Wenqiang Yu, Zhenjun Tang, Tutian Li, Yutong Yao, Siqiong Lu, Cewu |
| contents | Dexterous in-hand manipulation is a long-standing challenge in robotics due to complex contact dynamics and partial observability. While humans synergize vision and touch for such tasks, robotic approaches often prioritize one modality, therefore limiting adaptability. This paper introduces Flow Before Imitation (FBI), a visuotactile imitation learning framework that dynamically fuses tactile interactions with visual observations through motion dynamics. Unlike prior static fusion methods, FBI establishes a causal link between tactile signals and object motion via a dynamics-aware latent model. FBI employs a transformer-based interaction module to fuse flow-derived tactile features with visual inputs, training a one-step diffusion policy for real-time execution. Extensive experiments demonstrate that the proposed method outperforms the baseline methods in both simulation and the real world on two customized in-hand manipulation tasks and three standard dexterous manipulation tasks. Code, models, and more results are available in the website https://sites.google.com/view/dex-fbi. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_14441 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FBI: Learning Dexterous In-hand Manipulation with Dynamic Visuotactile Shortcut Policy Chen, Yijin Xu, Wenqiang Yu, Zhenjun Tang, Tutian Li, Yutong Yao, Siqiong Lu, Cewu Robotics Dexterous in-hand manipulation is a long-standing challenge in robotics due to complex contact dynamics and partial observability. While humans synergize vision and touch for such tasks, robotic approaches often prioritize one modality, therefore limiting adaptability. This paper introduces Flow Before Imitation (FBI), a visuotactile imitation learning framework that dynamically fuses tactile interactions with visual observations through motion dynamics. Unlike prior static fusion methods, FBI establishes a causal link between tactile signals and object motion via a dynamics-aware latent model. FBI employs a transformer-based interaction module to fuse flow-derived tactile features with visual inputs, training a one-step diffusion policy for real-time execution. Extensive experiments demonstrate that the proposed method outperforms the baseline methods in both simulation and the real world on two customized in-hand manipulation tasks and three standard dexterous manipulation tasks. Code, models, and more results are available in the website https://sites.google.com/view/dex-fbi. |
| title | FBI: Learning Dexterous In-hand Manipulation with Dynamic Visuotactile Shortcut Policy |
| topic | Robotics |
| url | https://arxiv.org/abs/2508.14441 |