EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a Wristband
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917696350715904 |
|---|---|
| author | Lee, Chi-Jung Zhang, Ruidong Agarwal, Devansh Yu, Tianhong Catherine Gunda, Vipin Lopez, Oliver Kim, James Yin, Sicheng Dong, Boao Li, Ke Sakashita, Mose Guimbretiere, Francois Zhang, Cheng |
| author_facet | Lee, Chi-Jung Zhang, Ruidong Agarwal, Devansh Yu, Tianhong Catherine Gunda, Vipin Lopez, Oliver Kim, James Yin, Sicheng Dong, Boao Li, Ke Sakashita, Mose Guimbretiere, Francois Zhang, Cheng |
| contents | Our hands serve as a fundamental means of interaction with the world around us. Therefore, understanding hand poses and interaction context is critical for human-computer interaction. We present EchoWrist, a low-power wristband that continuously estimates 3D hand pose and recognizes hand-object interactions using active acoustic sensing. EchoWrist is equipped with two speakers emitting inaudible sound waves toward the hand. These sound waves interact with the hand and its surroundings through reflections and diffractions, carrying rich information about the hand's shape and the objects it interacts with. The information captured by the two microphones goes through a deep learning inference system that recovers hand poses and identifies various everyday hand activities. Results from the two 12-participant user studies show that EchoWrist is effective and efficient at tracking 3D hand poses and recognizing hand-object interactions. Operating at 57.9mW, EchoWrist is able to continuously reconstruct 20 3D hand joints with MJEDE of 4.81mm and recognize 12 naturalistic hand-object interactions with 97.6% accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_17409 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a Wristband Lee, Chi-Jung Zhang, Ruidong Agarwal, Devansh Yu, Tianhong Catherine Gunda, Vipin Lopez, Oliver Kim, James Yin, Sicheng Dong, Boao Li, Ke Sakashita, Mose Guimbretiere, Francois Zhang, Cheng Human-Computer Interaction Our hands serve as a fundamental means of interaction with the world around us. Therefore, understanding hand poses and interaction context is critical for human-computer interaction. We present EchoWrist, a low-power wristband that continuously estimates 3D hand pose and recognizes hand-object interactions using active acoustic sensing. EchoWrist is equipped with two speakers emitting inaudible sound waves toward the hand. These sound waves interact with the hand and its surroundings through reflections and diffractions, carrying rich information about the hand's shape and the objects it interacts with. The information captured by the two microphones goes through a deep learning inference system that recovers hand poses and identifies various everyday hand activities. Results from the two 12-participant user studies show that EchoWrist is effective and efficient at tracking 3D hand poses and recognizing hand-object interactions. Operating at 57.9mW, EchoWrist is able to continuously reconstruct 20 3D hand joints with MJEDE of 4.81mm and recognize 12 naturalistic hand-object interactions with 97.6% accuracy. |
| title | EchoWrist: Continuous Hand Pose Tracking and Hand-Object Interaction Recognition Using Low-Power Active Acoustic Sensing On a Wristband |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2401.17409 |