SpellRing: Recognizing Continuous Fingerspelling in American Sign Language using a Ring
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929717506998272 |
|---|---|
| author | Lim, Hyunchul Dang, Nam Anh Lee, Dylan Yu, Tianhong Catherine Lu, Jane Li, Franklin Mingzhe Jin, Yiqi Ma, Yan Bi, Xiaojun Guimbretière, François Zhang, Cheng |
| author_facet | Lim, Hyunchul Dang, Nam Anh Lee, Dylan Yu, Tianhong Catherine Lu, Jane Li, Franklin Mingzhe Jin, Yiqi Ma, Yan Bi, Xiaojun Guimbretière, François Zhang, Cheng |
| contents | Fingerspelling is a critical part of American Sign Language (ASL) recognition and has become an accessible optional text entry method for Deaf and Hard of Hearing (DHH) individuals. In this paper, we introduce SpellRing, a single smart ring worn on the thumb that recognizes words continuously fingerspelled in ASL. SpellRing uses active acoustic sensing (via a microphone and speaker) and an inertial measurement unit (IMU) to track handshape and movement, which are processed through a deep learning algorithm using Connectionist Temporal Classification (CTC) loss. We evaluated the system with 20 ASL signers (13 fluent and 7 learners), using the MacKenzie-Soukoref Phrase Set of 1,164 words and 100 phrases. Offline evaluation yielded top-1 and top-5 word recognition accuracies of 82.45% (9.67%) and 92.42% (5.70%), respectively. In real-time, the system achieved a word error rate (WER) of 0.099 (0.039) on the phrases. Based on these results, we discuss key lessons and design implications for future minimally obtrusive ASL recognition wearables. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10830 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SpellRing: Recognizing Continuous Fingerspelling in American Sign Language using a Ring Lim, Hyunchul Dang, Nam Anh Lee, Dylan Yu, Tianhong Catherine Lu, Jane Li, Franklin Mingzhe Jin, Yiqi Ma, Yan Bi, Xiaojun Guimbretière, François Zhang, Cheng Human-Computer Interaction Fingerspelling is a critical part of American Sign Language (ASL) recognition and has become an accessible optional text entry method for Deaf and Hard of Hearing (DHH) individuals. In this paper, we introduce SpellRing, a single smart ring worn on the thumb that recognizes words continuously fingerspelled in ASL. SpellRing uses active acoustic sensing (via a microphone and speaker) and an inertial measurement unit (IMU) to track handshape and movement, which are processed through a deep learning algorithm using Connectionist Temporal Classification (CTC) loss. We evaluated the system with 20 ASL signers (13 fluent and 7 learners), using the MacKenzie-Soukoref Phrase Set of 1,164 words and 100 phrases. Offline evaluation yielded top-1 and top-5 word recognition accuracies of 82.45% (9.67%) and 92.42% (5.70%), respectively. In real-time, the system achieved a word error rate (WER) of 0.099 (0.039) on the phrases. Based on these results, we discuss key lessons and design implications for future minimally obtrusive ASL recognition wearables. |
| title | SpellRing: Recognizing Continuous Fingerspelling in American Sign Language using a Ring |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2502.10830 |