Parameter-efficient Adaptation of Multilingual Multimodal Models for Low-resource ASR
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916442814808064 |
|---|---|
| author | Gupta, Abhishek Parulekar, Amruta Chattopadhyay, Sameep Jyothi, Preethi |
| author_facet | Gupta, Abhishek Parulekar, Amruta Chattopadhyay, Sameep Jyothi, Preethi |
| contents | Automatic speech recognition (ASR) for low-resource languages remains a challenge due to the scarcity of labeled training data. Parameter-efficient fine-tuning and text-only adaptation are two popular methods that have been used to address such low-resource settings. In this work, we investigate how these techniques can be effectively combined using a multilingual multimodal model like SeamlessM4T. Multimodal models are able to leverage unlabeled text via text-only adaptation with further parameter-efficient ASR fine-tuning, thus boosting ASR performance. We also show cross-lingual transfer from a high-resource language, achieving up to a relative 17% WER reduction over a baseline in a zero-shot setting without any labeled speech. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13445 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Parameter-efficient Adaptation of Multilingual Multimodal Models for Low-resource ASR Gupta, Abhishek Parulekar, Amruta Chattopadhyay, Sameep Jyothi, Preethi Computation and Language Artificial Intelligence Machine Learning Audio and Speech Processing Automatic speech recognition (ASR) for low-resource languages remains a challenge due to the scarcity of labeled training data. Parameter-efficient fine-tuning and text-only adaptation are two popular methods that have been used to address such low-resource settings. In this work, we investigate how these techniques can be effectively combined using a multilingual multimodal model like SeamlessM4T. Multimodal models are able to leverage unlabeled text via text-only adaptation with further parameter-efficient ASR fine-tuning, thus boosting ASR performance. We also show cross-lingual transfer from a high-resource language, achieving up to a relative 17% WER reduction over a baseline in a zero-shot setting without any labeled speech. |
| title | Parameter-efficient Adaptation of Multilingual Multimodal Models for Low-resource ASR |
| topic | Computation and Language Artificial Intelligence Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2410.13445 |