American Sign Language to Text Translation using Transformer and Seq2Seq with LSTM
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917777693999104 |
|---|---|
| author | Putra, Gregorius Guntur Sunardi D'Layla, Adifa Widyadhani Chanda Wahono, Dimas Sarno, Riyanarto Haryono, Agus Tri |
| author_facet | Putra, Gregorius Guntur Sunardi D'Layla, Adifa Widyadhani Chanda Wahono, Dimas Sarno, Riyanarto Haryono, Agus Tri |
| contents | Sign language translation is one of the important issues in communication between deaf and hearing people, as it expresses words through hand, body, and mouth movements. American Sign Language is one of the sign languages used, one of which is the alphabetic sign. The development of neural machine translation technology is moving towards sign language translation. Transformer became the state-of-the-art in natural language processing. This study compares the Transformer with the Sequence-to-Sequence (Seq2Seq) model in translating sign language to text. In addition, an experiment was conducted by adding Residual Long Short-Term Memory (ResidualLSTM) in the Transformer. The addition of ResidualLSTM to the Transformer reduces the performance of the Transformer model by 23.37% based on the BLEU Score value. In comparison, the Transformer itself increases the BLEU Score value by 28.14 compared to the Seq2Seq model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10874 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | American Sign Language to Text Translation using Transformer and Seq2Seq with LSTM Putra, Gregorius Guntur Sunardi D'Layla, Adifa Widyadhani Chanda Wahono, Dimas Sarno, Riyanarto Haryono, Agus Tri Computation and Language Sign language translation is one of the important issues in communication between deaf and hearing people, as it expresses words through hand, body, and mouth movements. American Sign Language is one of the sign languages used, one of which is the alphabetic sign. The development of neural machine translation technology is moving towards sign language translation. Transformer became the state-of-the-art in natural language processing. This study compares the Transformer with the Sequence-to-Sequence (Seq2Seq) model in translating sign language to text. In addition, an experiment was conducted by adding Residual Long Short-Term Memory (ResidualLSTM) in the Transformer. The addition of ResidualLSTM to the Transformer reduces the performance of the Transformer model by 23.37% based on the BLEU Score value. In comparison, the Transformer itself increases the BLEU Score value by 28.14 compared to the Seq2Seq model. |
| title | American Sign Language to Text Translation using Transformer and Seq2Seq with LSTM |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.10874 |