The NGT200 Dataset: Geometric Multi-View Isolated Sign Recognition
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912041981181952 |
|---|---|
| author | Ranum, Oline Wessels, David R. Otterspeer, Gomer Bekkers, Erik J. Roelofsen, Floris Andersen, Jari I. |
| author_facet | Ranum, Oline Wessels, David R. Otterspeer, Gomer Bekkers, Erik J. Roelofsen, Floris Andersen, Jari I. |
| contents | Sign Language Processing (SLP) provides a foundation for a more inclusive future in language technology; however, the field faces several significant challenges that must be addressed to achieve practical, real-world applications. This work addresses multi-view isolated sign recognition (MV-ISR), and highlights the essential role of 3D awareness and geometry in SLP systems. We introduce the NGT200 dataset, a novel spatio-temporal multi-view benchmark, establishing MV-ISR as distinct from single-view ISR (SV-ISR). We demonstrate the benefits of synthetic data and propose conditioning sign representations on spatial symmetries inherent in sign language. Leveraging an SE(2) equivariant model improves MV-ISR performance by 8%-22% over the baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15284 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The NGT200 Dataset: Geometric Multi-View Isolated Sign Recognition Ranum, Oline Wessels, David R. Otterspeer, Gomer Bekkers, Erik J. Roelofsen, Floris Andersen, Jari I. Computer Vision and Pattern Recognition Computation and Language Sign Language Processing (SLP) provides a foundation for a more inclusive future in language technology; however, the field faces several significant challenges that must be addressed to achieve practical, real-world applications. This work addresses multi-view isolated sign recognition (MV-ISR), and highlights the essential role of 3D awareness and geometry in SLP systems. We introduce the NGT200 dataset, a novel spatio-temporal multi-view benchmark, establishing MV-ISR as distinct from single-view ISR (SV-ISR). We demonstrate the benefits of synthetic data and propose conditioning sign representations on spatial symmetries inherent in sign language. Leveraging an SE(2) equivariant model improves MV-ISR performance by 8%-22% over the baseline. |
| title | The NGT200 Dataset: Geometric Multi-View Isolated Sign Recognition |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2409.15284 |