Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2301.02310 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913183535464448 |
|---|---|
| author | Grady, Patrick Collins, Jeremy A. Tang, Chengcheng Twigg, Christopher D. Aneja, Kunal Hays, James Kemp, Charles C. |
| author_facet | Grady, Patrick Collins, Jeremy A. Tang, Chengcheng Twigg, Christopher D. Aneja, Kunal Hays, James Kemp, Charles C. |
| contents | Touch plays a fundamental role in manipulation for humans; however, machine perception of contact and pressure typically requires invasive sensors. Recent research has shown that deep models can estimate hand pressure based on a single RGB image. However, evaluations have been limited to controlled settings since collecting diverse data with ground-truth pressure measurements is difficult. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to apply pressure in a certain way, and this prompt can serve as a weak label to supervise models to perform well under varied conditions. We collect a novel dataset with 51 participants making fingertip contact with diverse objects. Our network, PressureVision++, outperforms human annotators and prior work. We also demonstrate an application of PressureVision++ to mixed reality where pressure estimation allows everyday surfaces to be used as arbitrary touch-sensitive interfaces. Code, data, and models are available online. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_02310 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images Grady, Patrick Collins, Jeremy A. Tang, Chengcheng Twigg, Christopher D. Aneja, Kunal Hays, James Kemp, Charles C. Computer Vision and Pattern Recognition Touch plays a fundamental role in manipulation for humans; however, machine perception of contact and pressure typically requires invasive sensors. Recent research has shown that deep models can estimate hand pressure based on a single RGB image. However, evaluations have been limited to controlled settings since collecting diverse data with ground-truth pressure measurements is difficult. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to apply pressure in a certain way, and this prompt can serve as a weak label to supervise models to perform well under varied conditions. We collect a novel dataset with 51 participants making fingertip contact with diverse objects. Our network, PressureVision++, outperforms human annotators and prior work. We also demonstrate an application of PressureVision++ to mixed reality where pressure estimation allows everyday surfaces to be used as arbitrary touch-sensitive interfaces. Code, data, and models are available online. |
| title | PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2301.02310 |