Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914955571232768 |
|---|---|
| author | Wang, Shiyao Zhao, Shiwan Zhou, Jiaming Kong, Aobo Qin, Yong |
| author_facet | Wang, Shiyao Zhao, Shiwan Zhou, Jiaming Kong, Aobo Qin, Yong |
| contents | Dysarthric speech recognition (DSR) presents a formidable challenge due to inherent inter-speaker variability, leading to severe performance degradation when applying DSR models to new dysarthric speakers. Traditional speaker adaptation methodologies typically involve fine-tuning models for each speaker, but this strategy is cost-prohibitive and inconvenient for disabled users, requiring substantial data collection. To address this issue, we introduce a prototype-based approach that markedly improves DSR performance for unseen dysarthric speakers without additional fine-tuning. Our method employs a feature extractor trained with HuBERT to produce per-word prototypes that encapsulate the characteristics of previously unseen speakers. These prototypes serve as the basis for classification. Additionally, we incorporate supervised contrastive learning to refine feature extraction. By enhancing representation quality, we further improve DSR performance, enabling effective personalized DSR. We release our code at https://github.com/NKU-HLT/PB-DSR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_18461 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation Wang, Shiyao Zhao, Shiwan Zhou, Jiaming Kong, Aobo Qin, Yong Sound Computation and Language Audio and Speech Processing Dysarthric speech recognition (DSR) presents a formidable challenge due to inherent inter-speaker variability, leading to severe performance degradation when applying DSR models to new dysarthric speakers. Traditional speaker adaptation methodologies typically involve fine-tuning models for each speaker, but this strategy is cost-prohibitive and inconvenient for disabled users, requiring substantial data collection. To address this issue, we introduce a prototype-based approach that markedly improves DSR performance for unseen dysarthric speakers without additional fine-tuning. Our method employs a feature extractor trained with HuBERT to produce per-word prototypes that encapsulate the characteristics of previously unseen speakers. These prototypes serve as the basis for classification. Additionally, we incorporate supervised contrastive learning to refine feature extraction. By enhancing representation quality, we further improve DSR performance, enabling effective personalized DSR. We release our code at https://github.com/NKU-HLT/PB-DSR. |
| title | Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.18461 |