SignTalk - A Deep Learning-Based System for Real-Time Sign Language Recognition & Voice Generation
Fuente:
Zenodo
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Recurso digital |
| Veröffentlicht: |
Zenodo
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901647193538560 |
|---|---|
| author | Gandu Lavanya Jaligam Srilekha Kare Ranjith Kumar Kelim Praveen Burra Manoj Kumar |
| author_facet | Gandu Lavanya Jaligam Srilekha Kare Ranjith Kumar Kelim Praveen Burra Manoj Kumar |
| contents | Sign language is an important and expressive method of communication used by individuals who are deaf or hard of hearing. However, communication difficulties often arise when interacting with people who do not understand sign language. To overcome this issue, this research presents SignTalk AI, a real-time sign language recognition system that converts hand gestures into readable text and spoken output using artificial intelligence and computer vision techniques. The proposed system utilizes a Convolutional Neural Network (CNN) trained on the American Sign Language (ASL) Alphabet Dataset to recognize hand gestures captured through a webcam. Video frames are processed continuously, and the region containing the hand gesture is extracted and prepared before being passed to the trained model for classification. To improve prediction stability, the system uses a frame-based prediction smoothing method along with confidence threshold filtering. In addition, MediaPipe hand detection is integrated so that predictions are generated only when a hand is detected in the frame, which helps reduce noise and improves system reliability. The recognized gestures are displayed as text and can also be converted into speech using text-to-speech technology. A web application built with Flask and OpenCV provides real-time visualization of gestures, prediction tracking, sentence generation, and voice output. Experimental results show that the system performs with high recognition accuracy and stable real-time performance under normal conditions. This solution can assist speech- and hearing-impaired individuals in communication and can be extended in the future to support continuous sign interpretation and mobile-based applications. |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19594547 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | SignTalk - A Deep Learning-Based System for Real-Time Sign Language Recognition & Voice Generation Gandu Lavanya Jaligam Srilekha Kare Ranjith Kumar Kelim Praveen Burra Manoj Kumar Sign Language Recognition Deep Learning CNN Computer Vision ASL Sign language is an important and expressive method of communication used by individuals who are deaf or hard of hearing. However, communication difficulties often arise when interacting with people who do not understand sign language. To overcome this issue, this research presents SignTalk AI, a real-time sign language recognition system that converts hand gestures into readable text and spoken output using artificial intelligence and computer vision techniques. The proposed system utilizes a Convolutional Neural Network (CNN) trained on the American Sign Language (ASL) Alphabet Dataset to recognize hand gestures captured through a webcam. Video frames are processed continuously, and the region containing the hand gesture is extracted and prepared before being passed to the trained model for classification. To improve prediction stability, the system uses a frame-based prediction smoothing method along with confidence threshold filtering. In addition, MediaPipe hand detection is integrated so that predictions are generated only when a hand is detected in the frame, which helps reduce noise and improves system reliability. The recognized gestures are displayed as text and can also be converted into speech using text-to-speech technology. A web application built with Flask and OpenCV provides real-time visualization of gestures, prediction tracking, sentence generation, and voice output. Experimental results show that the system performs with high recognition accuracy and stable real-time performance under normal conditions. This solution can assist speech- and hearing-impaired individuals in communication and can be extended in the future to support continuous sign interpretation and mobile-based applications. |
| title | SignTalk - A Deep Learning-Based System for Real-Time Sign Language Recognition & Voice Generation |
| topic | Sign Language Recognition Deep Learning CNN Computer Vision ASL |
| url | https://doi.org/10.5281/zenodo.19594547 |