Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929280460521472 |
|---|---|
| author | Huang, Kuan-Po Yang, Chih-Kai Fu, Yu-Kuan Dunbar, Ewan Lee, Hung-yi |
| author_facet | Huang, Kuan-Po Yang, Chih-Kai Fu, Yu-Kuan Dunbar, Ewan Lee, Hung-yi |
| contents | We introduce a new zero resource code-switched speech benchmark designed to directly assess the code-switching capabilities of self-supervised speech encoders. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can be assessed in a zero-resource manner. Our experiments encompass a variety of well-known speech encoders, including Wav2vec 2.0, HuBERT, XLSR, etc. We examine the impact of pre-training languages and model size on benchmark performance. Notably, though our results demonstrate that speech encoders with multilingual pre-training, exemplified by XLSR, outperform monolingual variants (Wav2vec 2.0, HuBERT) in code-switching scenarios, there is still substantial room for improvement in their code-switching linguistic abilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_03018 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages Huang, Kuan-Po Yang, Chih-Kai Fu, Yu-Kuan Dunbar, Ewan Lee, Hung-yi Audio and Speech Processing Computation and Language Sound We introduce a new zero resource code-switched speech benchmark designed to directly assess the code-switching capabilities of self-supervised speech encoders. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can be assessed in a zero-resource manner. Our experiments encompass a variety of well-known speech encoders, including Wav2vec 2.0, HuBERT, XLSR, etc. We examine the impact of pre-training languages and model size on benchmark performance. Notably, though our results demonstrate that speech encoders with multilingual pre-training, exemplified by XLSR, outperform monolingual variants (Wav2vec 2.0, HuBERT) in code-switching scenarios, there is still substantial room for improvement in their code-switching linguistic abilities. |
| title | Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2310.03018 |