Vision-Based Hand Gesture Customization from a Single Demonstration
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910629922603008 |
|---|---|
| author | Shahi, Soroush Mollyn, Vimal Park, Cori Tymoszek Kang, Richard Liberman, Asaf Levy, Oron Gong, Jun Bedri, Abdelkareem Laput, Gierad |
| author_facet | Shahi, Soroush Mollyn, Vimal Park, Cori Tymoszek Kang, Richard Liberman, Asaf Levy, Oron Gong, Jun Bedri, Abdelkareem Laput, Gierad |
| contents | Hand gesture recognition is becoming a more prevalent mode of human-computer interaction, especially as cameras proliferate across everyday devices. Despite continued progress in this field, gesture customization is often underexplored. Customization is crucial since it enables users to define and demonstrate gestures that are more natural, memorable, and accessible. However, customization requires efficient usage of user-provided data. We introduce a method that enables users to easily design bespoke gestures with a monocular camera from one demonstration. We employ transformers and meta-learning techniques to address few-shot learning challenges. Unlike prior work, our method supports any combination of one-handed, two-handed, static, and dynamic gestures, including different viewpoints, and the ability to handle irrelevant hand movements. We implement three real-world applications using our customization method, conduct a user study, and achieve up to 94% average recognition accuracy from one demonstration. Our work provides a viable path for vision-based gesture customization, laying the foundation for future advancements in this domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08420 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Vision-Based Hand Gesture Customization from a Single Demonstration Shahi, Soroush Mollyn, Vimal Park, Cori Tymoszek Kang, Richard Liberman, Asaf Levy, Oron Gong, Jun Bedri, Abdelkareem Laput, Gierad Human-Computer Interaction H.5.2; I.4 Hand gesture recognition is becoming a more prevalent mode of human-computer interaction, especially as cameras proliferate across everyday devices. Despite continued progress in this field, gesture customization is often underexplored. Customization is crucial since it enables users to define and demonstrate gestures that are more natural, memorable, and accessible. However, customization requires efficient usage of user-provided data. We introduce a method that enables users to easily design bespoke gestures with a monocular camera from one demonstration. We employ transformers and meta-learning techniques to address few-shot learning challenges. Unlike prior work, our method supports any combination of one-handed, two-handed, static, and dynamic gestures, including different viewpoints, and the ability to handle irrelevant hand movements. We implement three real-world applications using our customization method, conduct a user study, and achieve up to 94% average recognition accuracy from one demonstration. Our work provides a viable path for vision-based gesture customization, laying the foundation for future advancements in this domain. |
| title | Vision-Based Hand Gesture Customization from a Single Demonstration |
| topic | Human-Computer Interaction H.5.2; I.4 |
| url | https://arxiv.org/abs/2402.08420 |