Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.08780 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917775202582528 |
|---|---|
| author | Grimm, Jana Winkler, Miriam Kraus, Oliver Agustoslu, Tanalp |
| author_facet | Grimm, Jana Winkler, Miriam Kraus, Oliver Agustoslu, Tanalp |
| contents | This project explores methods to enhance sign language translation of German sign language, specifically focusing on disambiguation of homonyms. Sign language is ambiguous and understudied which is the basis for our experiments. We approach the improvement by training transformer-based models on various bodypart representations to shift the focus on said bodypart. To determine the impact of, e.g., the hand or mouth representations, we experiment with different combinations. The results show that focusing on the mouth increases the performance in small dataset settings while shifting the focus on the hands retrieves better results in larger dataset settings. Our results contribute to better accessibility for non-hearing persons by improving the systems powering digital assistants, enabling a more accurate interaction. The code for this project can be found on GitHub. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_08780 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sign Language Sense Disambiguation Grimm, Jana Winkler, Miriam Kraus, Oliver Agustoslu, Tanalp Computation and Language This project explores methods to enhance sign language translation of German sign language, specifically focusing on disambiguation of homonyms. Sign language is ambiguous and understudied which is the basis for our experiments. We approach the improvement by training transformer-based models on various bodypart representations to shift the focus on said bodypart. To determine the impact of, e.g., the hand or mouth representations, we experiment with different combinations. The results show that focusing on the mouth increases the performance in small dataset settings while shifting the focus on the hands retrieves better results in larger dataset settings. Our results contribute to better accessibility for non-hearing persons by improving the systems powering digital assistants, enabling a more accurate interaction. The code for this project can be found on GitHub. |
| title | Sign Language Sense Disambiguation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.08780 |