Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.07842 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929382680952832 |
|---|---|
| author | Khassanov, Yerbolat Chen, Zhipeng Chen, Tianfeng Chong, Tze Yuang Li, Wei Zhang, Jun Lu, Lu Wang, Yuxuan |
| author_facet | Khassanov, Yerbolat Chen, Zhipeng Chen, Tianfeng Chong, Tze Yuang Li, Wei Zhang, Jun Lu, Lu Wang, Yuxuan |
| contents | This paper addresses challenges in integrating new languages into a pre-trained multilingual automatic speech recognition (mASR) system, particularly in scenarios where training data for existing languages is limited or unavailable. The proposed method employs a dual-pipeline with low-rank adaptation (LoRA). It maintains two data flow pipelines-one for existing languages and another for new languages. The primary pipeline follows the standard flow through the pre-trained parameters of mASR, while the secondary pipeline additionally utilizes language-specific parameters represented by LoRA and a separate output decoder module. Importantly, the proposed approach minimizes the performance degradation of existing languages and enables a language-agnostic operation mode, facilitated by a decoder selection strategy. We validate the effectiveness of the proposed method by extending the pre-trained Whisper model to 19 new languages from the FLEURS dataset |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07842 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR Khassanov, Yerbolat Chen, Zhipeng Chen, Tianfeng Chong, Tze Yuang Li, Wei Zhang, Jun Lu, Lu Wang, Yuxuan Audio and Speech Processing Computation and Language This paper addresses challenges in integrating new languages into a pre-trained multilingual automatic speech recognition (mASR) system, particularly in scenarios where training data for existing languages is limited or unavailable. The proposed method employs a dual-pipeline with low-rank adaptation (LoRA). It maintains two data flow pipelines-one for existing languages and another for new languages. The primary pipeline follows the standard flow through the pre-trained parameters of mASR, while the secondary pipeline additionally utilizes language-specific parameters represented by LoRA and a separate output decoder module. Importantly, the proposed approach minimizes the performance degradation of existing languages and enables a language-agnostic operation mode, facilitated by a decoder selection strategy. We validate the effectiveness of the proposed method by extending the pre-trained Whisper model to 19 new languages from the FLEURS dataset |
| title | Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR |
| topic | Audio and Speech Processing Computation and Language |
| url | https://arxiv.org/abs/2406.07842 |