Feel the Force: Contact-Driven Learning from Humans
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918043199733760 |
|---|---|
| author | Adeniji, Ademi Chen, Zhuoran Liu, Vincent Pattabiraman, Venkatesh Bhirangi, Raunaq Haldar, Siddhant Abbeel, Pieter Pinto, Lerrel |
| author_facet | Adeniji, Ademi Chen, Zhuoran Liu, Vincent Pattabiraman, Venkatesh Bhirangi, Raunaq Haldar, Siddhant Abbeel, Pieter Pinto, Lerrel |
| contents | Controlling fine-grained forces during manipulation remains a core challenge in robotics. While robot policies learned from robot-collected data or simulation show promise, they struggle to generalize across the diverse range of real-world interactions. Learning directly from humans offers a scalable solution, enabling demonstrators to perform skills in their natural embodiment and in everyday environments. However, visual demonstrations alone lack the information needed to infer precise contact forces. We present FeelTheForce (FTF): a robot learning system that models human tactile behavior to learn force-sensitive manipulation. Using a tactile glove to measure contact forces and a vision-based model to estimate hand pose, we train a closed-loop policy that continuously predicts the forces needed for manipulation. This policy is re-targeted to a Franka Panda robot with tactile gripper sensors using shared visual and action representations. At execution, a PD controller modulates gripper closure to track predicted forces-enabling precise, force-aware control. Our approach grounds robust low-level force control in scalable human supervision, achieving a 77% success rate across 5 force-sensitive manipulation tasks. Code and videos are available at https://feel-the-force-ftf.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_01944 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Feel the Force: Contact-Driven Learning from Humans Adeniji, Ademi Chen, Zhuoran Liu, Vincent Pattabiraman, Venkatesh Bhirangi, Raunaq Haldar, Siddhant Abbeel, Pieter Pinto, Lerrel Robotics Artificial Intelligence Controlling fine-grained forces during manipulation remains a core challenge in robotics. While robot policies learned from robot-collected data or simulation show promise, they struggle to generalize across the diverse range of real-world interactions. Learning directly from humans offers a scalable solution, enabling demonstrators to perform skills in their natural embodiment and in everyday environments. However, visual demonstrations alone lack the information needed to infer precise contact forces. We present FeelTheForce (FTF): a robot learning system that models human tactile behavior to learn force-sensitive manipulation. Using a tactile glove to measure contact forces and a vision-based model to estimate hand pose, we train a closed-loop policy that continuously predicts the forces needed for manipulation. This policy is re-targeted to a Franka Panda robot with tactile gripper sensors using shared visual and action representations. At execution, a PD controller modulates gripper closure to track predicted forces-enabling precise, force-aware control. Our approach grounds robust low-level force control in scalable human supervision, achieving a 77% success rate across 5 force-sensitive manipulation tasks. Code and videos are available at https://feel-the-force-ftf.github.io. |
| title | Feel the Force: Contact-Driven Learning from Humans |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2506.01944 |