Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914056902803456 |
|---|---|
| author | Li, Chin-Jou Yeo, Eunjung Choi, Kwanghee Pérez-Toro, Paula Andrea Someki, Masao Das, Rohan Kumar Yue, Zhengjun Orozco-Arroyave, Juan Rafael Nöth, Elmar Mortensen, David R. |
| author_facet | Li, Chin-Jou Yeo, Eunjung Choi, Kwanghee Pérez-Toro, Paula Andrea Someki, Masao Das, Rohan Kumar Yue, Zhengjun Orozco-Arroyave, Juan Rafael Nöth, Elmar Mortensen, David R. |
| contents | Automatic speech recognition (ASR) for dysarthric speech remains challenging due to data scarcity, particularly in non-English languages. To address this, we fine-tune a voice conversion model on English dysarthric speech (UASpeech) to encode both speaker characteristics and prosodic distortions, then apply it to convert healthy non-English speech (FLEURS) into non-English dysarthric-like speech. The generated data is then used to fine-tune a multilingual ASR model, Massively Multilingual Speech (MMS), for improved dysarthric speech recognition. Evaluation on PC-GITA (Spanish), EasyCall (Italian), and SSNCE (Tamil) demonstrates that VC with both speaker and prosody conversion significantly outperforms the off-the-shelf MMS performance and conventional augmentation techniques such as speed and tempo perturbation. Objective and subjective analyses of the generated data further confirm that the generated speech simulates dysarthric characteristics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14874 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages Li, Chin-Jou Yeo, Eunjung Choi, Kwanghee Pérez-Toro, Paula Andrea Someki, Masao Das, Rohan Kumar Yue, Zhengjun Orozco-Arroyave, Juan Rafael Nöth, Elmar Mortensen, David R. Computation and Language Sound Audio and Speech Processing Automatic speech recognition (ASR) for dysarthric speech remains challenging due to data scarcity, particularly in non-English languages. To address this, we fine-tune a voice conversion model on English dysarthric speech (UASpeech) to encode both speaker characteristics and prosodic distortions, then apply it to convert healthy non-English speech (FLEURS) into non-English dysarthric-like speech. The generated data is then used to fine-tune a multilingual ASR model, Massively Multilingual Speech (MMS), for improved dysarthric speech recognition. Evaluation on PC-GITA (Spanish), EasyCall (Italian), and SSNCE (Tamil) demonstrates that VC with both speaker and prosody conversion significantly outperforms the off-the-shelf MMS performance and conventional augmentation techniques such as speed and tempo perturbation. Objective and subjective analyses of the generated data further confirm that the generated speech simulates dysarthric characteristics. |
| title | Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.14874 |