Self-supervised perception for tactile skin covered dexterous hands
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910948002889728 |
|---|---|
| author | Sharma, Akash Higuera, Carolina Bodduluri, Chaithanya Krishna Liu, Zixi Fan, Taosha Hellebrekers, Tess Lambeta, Mike Boots, Byron Kaess, Michael Wu, Tingfan Hogan, Francois Robert Mukadam, Mustafa |
| author_facet | Sharma, Akash Higuera, Carolina Bodduluri, Chaithanya Krishna Liu, Zixi Fan, Taosha Hellebrekers, Tess Lambeta, Mike Boots, Byron Kaess, Michael Wu, Tingfan Hogan, Francois Robert Mukadam, Mustafa |
| contents | We present Sparsh-skin, a pre-trained encoder for magnetic skin sensors distributed across the fingertips, phalanges, and palm of a dexterous robot hand. Magnetic tactile skins offer a flexible form factor for hand-wide coverage with fast response times, in contrast to vision-based tactile sensors that are restricted to the fingertips and limited by bandwidth. Full hand tactile perception is crucial for robot dexterity. However, a lack of general-purpose models, challenges with interpreting magnetic flux and calibration have limited the adoption of these sensors. Sparsh-skin, given a history of kinematic and tactile sensing across a hand, outputs a latent tactile embedding that can be used in any downstream task. The encoder is self-supervised via self-distillation on a variety of unlabeled hand-object interactions using an Allegro hand sensorized with Xela uSkin. In experiments across several benchmark tasks, from state estimation to policy learning, we find that pretrained Sparsh-skin representations are both sample efficient in learning downstream tasks and improve task performance by over 41% compared to prior work and over 56% compared to end-to-end learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11420 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Self-supervised perception for tactile skin covered dexterous hands Sharma, Akash Higuera, Carolina Bodduluri, Chaithanya Krishna Liu, Zixi Fan, Taosha Hellebrekers, Tess Lambeta, Mike Boots, Byron Kaess, Michael Wu, Tingfan Hogan, Francois Robert Mukadam, Mustafa Robotics We present Sparsh-skin, a pre-trained encoder for magnetic skin sensors distributed across the fingertips, phalanges, and palm of a dexterous robot hand. Magnetic tactile skins offer a flexible form factor for hand-wide coverage with fast response times, in contrast to vision-based tactile sensors that are restricted to the fingertips and limited by bandwidth. Full hand tactile perception is crucial for robot dexterity. However, a lack of general-purpose models, challenges with interpreting magnetic flux and calibration have limited the adoption of these sensors. Sparsh-skin, given a history of kinematic and tactile sensing across a hand, outputs a latent tactile embedding that can be used in any downstream task. The encoder is self-supervised via self-distillation on a variety of unlabeled hand-object interactions using an Allegro hand sensorized with Xela uSkin. In experiments across several benchmark tasks, from state estimation to policy learning, we find that pretrained Sparsh-skin representations are both sample efficient in learning downstream tasks and improve task performance by over 41% compared to prior work and over 56% compared to end-to-end learning. |
| title | Self-supervised perception for tactile skin covered dexterous hands |
| topic | Robotics |
| url | https://arxiv.org/abs/2505.11420 |