Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2411.18266 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915196924067840 |
|---|---|
| author | Tang, Chenyu Gao, Shuo Li, Cong Yi, Wentian Jin, Yuxuan Zhai, Xiaoxue Lei, Sixuan Meng, Hongbei Zhang, Zibo Xu, Muzi Wang, Shengbo Chen, Xuhang Wang, Chenxi Yang, Hongyun Wang, Ningli Wang, Wenyu Cao, Jin Feng, Xiaodong Smielewski, Peter Pan, Yu Song, Wenhui Birchall, Martin Occhipinti, Luigi G. |
| author_facet | Tang, Chenyu Gao, Shuo Li, Cong Yi, Wentian Jin, Yuxuan Zhai, Xiaoxue Lei, Sixuan Meng, Hongbei Zhang, Zibo Xu, Muzi Wang, Shengbo Chen, Xuhang Wang, Chenxi Yang, Hongyun Wang, Ningli Wang, Wenyu Cao, Jin Feng, Xiaodong Smielewski, Peter Pan, Yu Song, Wenhui Birchall, Martin Occhipinti, Luigi G. |
| contents | Wearable silent speech systems hold significant potential for restoring communication in patients with speech impairments. However, seamless, coherent speech remains elusive, and clinical efficacy is still unproven. Here, we present an AI-driven intelligent throat (IT) system that integrates throat muscle vibrations and carotid pulse signal sensors with large language model (LLM) processing to enable fluent, emotionally expressive communication. The system utilizes ultrasensitive textile strain sensors to capture high-quality signals from the neck area and supports token-level processing for real-time, continuous speech decoding, enabling seamless, delay-free communication. In tests with five stroke patients with dysarthria, IT's LLM agents intelligently corrected token errors and enriched sentence-level emotional and logical coherence, achieving low error rates (4.2% word error rate, 2.9% sentence error rate) and a 55% increase in user satisfaction. This work establishes a portable, intuitive communication platform for patients with dysarthria with the potential to be applied broadly across different neurological conditions and in multi-language support systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18266 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Wearable intelligent throat enables natural speech in stroke patients with dysarthria Tang, Chenyu Gao, Shuo Li, Cong Yi, Wentian Jin, Yuxuan Zhai, Xiaoxue Lei, Sixuan Meng, Hongbei Zhang, Zibo Xu, Muzi Wang, Shengbo Chen, Xuhang Wang, Chenxi Yang, Hongyun Wang, Ningli Wang, Wenyu Cao, Jin Feng, Xiaodong Smielewski, Peter Pan, Yu Song, Wenhui Birchall, Martin Occhipinti, Luigi G. Audio and Speech Processing Artificial Intelligence Sound Systems and Control Wearable silent speech systems hold significant potential for restoring communication in patients with speech impairments. However, seamless, coherent speech remains elusive, and clinical efficacy is still unproven. Here, we present an AI-driven intelligent throat (IT) system that integrates throat muscle vibrations and carotid pulse signal sensors with large language model (LLM) processing to enable fluent, emotionally expressive communication. The system utilizes ultrasensitive textile strain sensors to capture high-quality signals from the neck area and supports token-level processing for real-time, continuous speech decoding, enabling seamless, delay-free communication. In tests with five stroke patients with dysarthria, IT's LLM agents intelligently corrected token errors and enriched sentence-level emotional and logical coherence, achieving low error rates (4.2% word error rate, 2.9% sentence error rate) and a 55% increase in user satisfaction. This work establishes a portable, intuitive communication platform for patients with dysarthria with the potential to be applied broadly across different neurological conditions and in multi-language support systems. |
| title | Wearable intelligent throat enables natural speech in stroke patients with dysarthria |
| topic | Audio and Speech Processing Artificial Intelligence Sound Systems and Control |
| url | https://arxiv.org/abs/2411.18266 |