Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929515131830272 |
|---|---|
| author | Piñeiro-Martín, Andrés García-Mateo, Carmen Docío-Fernández, Laura López-Pérez, María del Carmen Rehm, Georg |
| author_facet | Piñeiro-Martín, Andrés García-Mateo, Carmen Docío-Fernández, Laura López-Pérez, María del Carmen Rehm, Georg |
| contents | This paper addresses the challenge of integrating low-resource languages into multilingual automatic speech recognition (ASR) systems. We introduce a novel application of weighted cross-entropy, typically used for unbalanced datasets, to facilitate the integration of low-resource languages into pre-trained multilingual ASR models within the context of continual multilingual learning. We fine-tune the Whisper multilingual ASR model on five high-resource languages and one low-resource language, employing language-weighted dynamic cross-entropy and data augmentation. The results show a remarkable 6.69% word error rate (WER) reduction for the low-resource language compared to the fine-tuned model without applying our approach, and a 48.86% WER reduction compared to the original Whisper model. In addition, our approach yields an average WER reduction of 3.29% across the six languages, showing no degradation for the high-resource languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16954 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition Piñeiro-Martín, Andrés García-Mateo, Carmen Docío-Fernández, Laura López-Pérez, María del Carmen Rehm, Georg Computation and Language Sound Audio and Speech Processing This paper addresses the challenge of integrating low-resource languages into multilingual automatic speech recognition (ASR) systems. We introduce a novel application of weighted cross-entropy, typically used for unbalanced datasets, to facilitate the integration of low-resource languages into pre-trained multilingual ASR models within the context of continual multilingual learning. We fine-tune the Whisper multilingual ASR model on five high-resource languages and one low-resource language, employing language-weighted dynamic cross-entropy and data augmentation. The results show a remarkable 6.69% word error rate (WER) reduction for the low-resource language compared to the fine-tuned model without applying our approach, and a 48.86% WER reduction compared to the original Whisper model. In addition, our approach yields an average WER reduction of 3.29% across the six languages, showing no degradation for the high-resource languages. |
| title | Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2409.16954 |