SIGN-AI: An AI Assistant for Sign Language
Fuente:
Zenodo
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901599263129600 |
|---|---|
| author | Guechi Ayoub |
| author_facet | Guechi Ayoub |
| contents | <p>This paper introduces SIGN-AI, a unified framework for high-fidelity sign language synthesis<br>via text, voice, and video Inputs, it is a transformative framework for real-time sign language <br>synthesis addressing critical accessibility gaps for 70 million deaf individuals worldwide.<br>Confronting significant barriers—including 83% lack of accessible STEM education materials,<br>only 4.7% online content with sign interpretation, and 3× higher unemployment rates—we<br>propose two novel algorithmic approaches: Algorithm 1 (Text/Image-Driven Synthesis)<br>employs a modular pipeline converting multimodal inputs to text, generating sign images<br>via Vision Transformers trained on SGN-IMG datasets, animating frames through optical<br>flow prediction (0.5-2s clips), and sequencing with LSTM models. This approach delivers<br>computational efficiency (CPU-feasible) and explicit sign control. Algorithm 2 (End-to-<br>End Video Synthesis) utilizes diffusion models trained on SGN-VID datasets to directly<br>translate text into motion vectors, producing 1-3s sign segments with attention-based fusion,<br>achieving superior motion fidelity through learned co-articulation.Benchmarks demonstrate<br>70% resource reduction (Algorithm 1) and 92% motion naturalness (Algorithm 2) versus state-<br>of-the-art. Projected impacts include: 45% faster STEM concept acquisition, 30% increase<br>in deaf graduates by 2035, and $17B annual economic productivity gain. The framework<br>enables real-time translation of streaming media, educational content, and professional<br>communications, with future research targeting low-latency video-to-video conversion and<br>an AI Assistant.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_15741440 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | SIGN-AI: An AI Assistant for Sign Language Guechi Ayoub Sign Language Synthesis Multimodal AI Accessibility Technology Educational Inclusion Deaf Commu- nication AI Video AI Assistant <p>This paper introduces SIGN-AI, a unified framework for high-fidelity sign language synthesis<br>via text, voice, and video Inputs, it is a transformative framework for real-time sign language <br>synthesis addressing critical accessibility gaps for 70 million deaf individuals worldwide.<br>Confronting significant barriers—including 83% lack of accessible STEM education materials,<br>only 4.7% online content with sign interpretation, and 3× higher unemployment rates—we<br>propose two novel algorithmic approaches: Algorithm 1 (Text/Image-Driven Synthesis)<br>employs a modular pipeline converting multimodal inputs to text, generating sign images<br>via Vision Transformers trained on SGN-IMG datasets, animating frames through optical<br>flow prediction (0.5-2s clips), and sequencing with LSTM models. This approach delivers<br>computational efficiency (CPU-feasible) and explicit sign control. Algorithm 2 (End-to-<br>End Video Synthesis) utilizes diffusion models trained on SGN-VID datasets to directly<br>translate text into motion vectors, producing 1-3s sign segments with attention-based fusion,<br>achieving superior motion fidelity through learned co-articulation.Benchmarks demonstrate<br>70% resource reduction (Algorithm 1) and 92% motion naturalness (Algorithm 2) versus state-<br>of-the-art. Projected impacts include: 45% faster STEM concept acquisition, 30% increase<br>in deaf graduates by 2035, and $17B annual economic productivity gain. The framework<br>enables real-time translation of streaming media, educational content, and professional<br>communications, with future research targeting low-latency video-to-video conversion and<br>an AI Assistant.</p> |
| title | SIGN-AI: An AI Assistant for Sign Language |
| topic | Sign Language Synthesis Multimodal AI Accessibility Technology Educational Inclusion Deaf Commu- nication AI Video AI Assistant |
| url | https://doi.org/10.5281/zenodo.15741440 |