PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913183535464448 |
|---|---|
| author | Grady, Patrick Collins, Jeremy A. Tang, Chengcheng Twigg, Christopher D. Aneja, Kunal Hays, James Kemp, Charles C. |
| author_facet | Grady, Patrick Collins, Jeremy A. Tang, Chengcheng Twigg, Christopher D. Aneja, Kunal Hays, James Kemp, Charles C. |
| contents | Touch plays a fundamental role in manipulation for humans; however, machine perception of contact and pressure typically requires invasive sensors. Recent research has shown that deep models can estimate hand pressure based on a single RGB image. However, evaluations have been limited to controlled settings since collecting diverse data with ground-truth pressure measurements is difficult. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to apply pressure in a certain way, and this prompt can serve as a weak label to supervise models to perform well under varied conditions. We collect a novel dataset with 51 participants making fingertip contact with diverse objects. Our network, PressureVision++, outperforms human annotators and prior work. We also demonstrate an application of PressureVision++ to mixed reality where pressure estimation allows everyday surfaces to be used as arbitrary touch-sensitive interfaces. Code, data, and models are available online. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_02310 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images Grady, Patrick Collins, Jeremy A. Tang, Chengcheng Twigg, Christopher D. Aneja, Kunal Hays, James Kemp, Charles C. Computer Vision and Pattern Recognition Touch plays a fundamental role in manipulation for humans; however, machine perception of contact and pressure typically requires invasive sensors. Recent research has shown that deep models can estimate hand pressure based on a single RGB image. However, evaluations have been limited to controlled settings since collecting diverse data with ground-truth pressure measurements is difficult. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to apply pressure in a certain way, and this prompt can serve as a weak label to supervise models to perform well under varied conditions. We collect a novel dataset with 51 participants making fingertip contact with diverse objects. Our network, PressureVision++, outperforms human annotators and prior work. We also demonstrate an application of PressureVision++ to mixed reality where pressure estimation allows everyday surfaces to be used as arbitrary touch-sensitive interfaces. Code, data, and models are available online. |
| title | PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2301.02310 |