Multimodal Indian Sign Language Translator using Gesture Recognition and Voice Input
Fuente:
Zenodo
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Recurso digital |
| Pubblicazione: |
Zenodo
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866901963535286272 |
|---|---|
| author | Dr. A. V. Sable Purvesh P. Savalakhe Mayuri S. Pache Om S. Naringe Shrawani S. Bonde Harsh S. Kapile |
| author_facet | Dr. A. V. Sable Purvesh P. Savalakhe Mayuri S. Pache Om S. Naringe Shrawani S. Bonde Harsh S. Kapile |
| contents | Abstract - In this paper, we present the design and implementation of a most people do not understand sign language, which creates a gap between them and normal users. Sign Language Detection System aimed at improving communication between hearing-impaired individuals and the general population. Communication barriers often arise due to the lack of understanding of Indian Sign Language (ISL), making it difficult for non-sign language users to interact effectively. To address this issue, the proposed system provides a multimodal communication platform that enables users to interact using text, voice, and hand gestures. The system is designed with multiple functionalities, including text-to-sign conversion, sign-to-text recognition, and voice-to-sign translation, along with an integrated messaging feature. Users can send and receive messages in both textual and sign-based formats, enhancing accessibility and usability. The text-to-sign module converts user-input text into a sequence of corresponding ISL alphabet images, allowing users to visually interpret words in sign language. The voice-to-sign module utilizes speech recognition techniques to convert spoken input into text, which is further processed into sign representations. The sign-to-text module captures real-time hand gestures through a camera and translates them into textual output, enabling two-way communication. For the implementation of the system, we used Python and Django as the backend framework to handle application logic and data processing, while HTML, CSS, and Bootstrap were used to develop a responsive and user-friendly interface. For gesture recognition, MediaPipe is employed to detect and extract hand landmarks in real-time. These landmarks are then used as input features for a Random Forest machine learning algorithm, which classifies gestures into corresponding alphabets or words. Overall, the proposed Sign Language Detection System demonstrates an effective integration of machine learning, computer vision, and web technologies to create A practical and accessible solution for ISL communication. The system has the potential to bridge communication gaps and promote inclusivity for the hearing-impaired community. |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19511544 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Multimodal Indian Sign Language Translator using Gesture Recognition and Voice Input Dr. A. V. Sable Purvesh P. Savalakhe Mayuri S. Pache Om S. Naringe Shrawani S. Bonde Harsh S. Kapile Indian Sign Language (ISL) Gesture Recognition Media-Pipe Random Forest Algorithm Sign-to-Text Conversion Voice-to-Sign Translation Real-Time Communication Abstract - In this paper, we present the design and implementation of a most people do not understand sign language, which creates a gap between them and normal users. Sign Language Detection System aimed at improving communication between hearing-impaired individuals and the general population. Communication barriers often arise due to the lack of understanding of Indian Sign Language (ISL), making it difficult for non-sign language users to interact effectively. To address this issue, the proposed system provides a multimodal communication platform that enables users to interact using text, voice, and hand gestures. The system is designed with multiple functionalities, including text-to-sign conversion, sign-to-text recognition, and voice-to-sign translation, along with an integrated messaging feature. Users can send and receive messages in both textual and sign-based formats, enhancing accessibility and usability. The text-to-sign module converts user-input text into a sequence of corresponding ISL alphabet images, allowing users to visually interpret words in sign language. The voice-to-sign module utilizes speech recognition techniques to convert spoken input into text, which is further processed into sign representations. The sign-to-text module captures real-time hand gestures through a camera and translates them into textual output, enabling two-way communication. For the implementation of the system, we used Python and Django as the backend framework to handle application logic and data processing, while HTML, CSS, and Bootstrap were used to develop a responsive and user-friendly interface. For gesture recognition, MediaPipe is employed to detect and extract hand landmarks in real-time. These landmarks are then used as input features for a Random Forest machine learning algorithm, which classifies gestures into corresponding alphabets or words. Overall, the proposed Sign Language Detection System demonstrates an effective integration of machine learning, computer vision, and web technologies to create A practical and accessible solution for ISL communication. The system has the potential to bridge communication gaps and promote inclusivity for the hearing-impaired community. |
| title | Multimodal Indian Sign Language Translator using Gesture Recognition and Voice Input |
| topic | Indian Sign Language (ISL) Gesture Recognition Media-Pipe Random Forest Algorithm Sign-to-Text Conversion Voice-to-Sign Translation Real-Time Communication |
| url | https://doi.org/10.5281/zenodo.19511544 |