Sign Language Recognition Based On Facial Expression and Hand Skeleton
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929406326341632 |
|---|---|
| author | Long, Zhiyu Liu, Xingyou Qiao, Jiaqi Li, Zhi |
| author_facet | Long, Zhiyu Liu, Xingyou Qiao, Jiaqi Li, Zhi |
| contents | Sign language is a visual language used by the deaf and dumb community to communicate. However, for most recognition methods based on monocular cameras, the recognition accuracy is low and the robustness is poor. Even if the effect is good on some data, it may perform poorly in other data with different interference due to the inability to extract effective features. To solve these problems, we propose a sign language recognition network that integrates skeleton features of hands and facial expression. Among this, we propose a hand skeleton feature extraction based on coordinate transformation to describe the shape of the hand more accurately. Moreover, by incorporating facial expression information, the accuracy and robustness of sign language recognition are finally improved, which was verified on A Dataset for Argentinian Sign Language and SEU's Chinese Sign Language Recognition Database (SEUCSLRD). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02241 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sign Language Recognition Based On Facial Expression and Hand Skeleton Long, Zhiyu Liu, Xingyou Qiao, Jiaqi Li, Zhi Computer Vision and Pattern Recognition Sign language is a visual language used by the deaf and dumb community to communicate. However, for most recognition methods based on monocular cameras, the recognition accuracy is low and the robustness is poor. Even if the effect is good on some data, it may perform poorly in other data with different interference due to the inability to extract effective features. To solve these problems, we propose a sign language recognition network that integrates skeleton features of hands and facial expression. Among this, we propose a hand skeleton feature extraction based on coordinate transformation to describe the shape of the hand more accurately. Moreover, by incorporating facial expression information, the accuracy and robustness of sign language recognition are finally improved, which was verified on A Dataset for Argentinian Sign Language and SEU's Chinese Sign Language Recognition Database (SEUCSLRD). |
| title | Sign Language Recognition Based On Facial Expression and Hand Skeleton |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.02241 |