Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.16842 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908720357703680 |
|---|---|
| author | Song, Yuxin Ray Li, Jinzhou Fu, Rao Murphy, Devin Zhou, Kaichen Shiv, Rishi Li, Yaqi Xiong, Haoyu Owens, Crystal Elaine Du, Yilun Luo, Yiyue Cheng, Xianyi Torralba, Antonio Matusik, Wojciech Liang, Paul Pu |
| author_facet | Song, Yuxin Ray Li, Jinzhou Fu, Rao Murphy, Devin Zhou, Kaichen Shiv, Rishi Li, Yaqi Xiong, Haoyu Owens, Crystal Elaine Du, Yilun Luo, Yiyue Cheng, Xianyi Torralba, Antonio Matusik, Wojciech Liang, Paul Pu |
| contents | The human hand is our primary interface to the physical world, yet egocentric perception rarely knows when, where, or how forcefully it makes contact. Robust wearable tactile sensors are scarce, and no existing in-the-wild datasets align first-person video with full-hand touch. To bridge the gap between visual perception and physical interaction, we present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset, containing 5.1 hours of synchronized video-touch-pose data and 2,900 curated clips with detailed text annotations. Using OpenTouch, we introduce retrieval and classification benchmarks that probe how touch grounds perception and action. We show that tactile signals provide a compact yet powerful cue for grasp understanding, strengthen cross-modal alignment, and can be reliably retrieved from in-the-wild video queries. By releasing this annotated vision-touch-pose dataset and benchmark, we aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16842 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction Song, Yuxin Ray Li, Jinzhou Fu, Rao Murphy, Devin Zhou, Kaichen Shiv, Rishi Li, Yaqi Xiong, Haoyu Owens, Crystal Elaine Du, Yilun Luo, Yiyue Cheng, Xianyi Torralba, Antonio Matusik, Wojciech Liang, Paul Pu Computer Vision and Pattern Recognition Artificial Intelligence Robotics The human hand is our primary interface to the physical world, yet egocentric perception rarely knows when, where, or how forcefully it makes contact. Robust wearable tactile sensors are scarce, and no existing in-the-wild datasets align first-person video with full-hand touch. To bridge the gap between visual perception and physical interaction, we present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset, containing 5.1 hours of synchronized video-touch-pose data and 2,900 curated clips with detailed text annotations. Using OpenTouch, we introduce retrieval and classification benchmarks that probe how touch grounds perception and action. We show that tactile signals provide a compact yet powerful cue for grasp understanding, strengthen cross-modal alignment, and can be reliably retrieved from in-the-wild video queries. By releasing this annotated vision-touch-pose dataset and benchmark, we aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation. |
| title | OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2512.16842 |