Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.12501 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916575907414016 |
|---|---|
| author | Tran, Minh Pang, Yutong Paul, Debjyoti Pandey, Laxmi Jiang, Kevin Guo, Jinxi Li, Ke Zhang, Shun Zhang, Xuedong Lei, Xin |
| author_facet | Tran, Minh Pang, Yutong Paul, Debjyoti Pandey, Laxmi Jiang, Kevin Guo, Jinxi Li, Ke Zhang, Shun Zhang, Xuedong Lei, Xin |
| contents | We introduce DAS (Domain Adaptation with Synthetic data), a novel domain adaptation framework for pre-trained ASR model, designed to efficiently adapt to various language-defined domains without requiring any real data. In particular, DAS first prompts large language models (LLMs) to generate domain-specific texts before converting these texts to speech via text-to-speech technology. The synthetic data is used to fine-tune Whisper with Low-Rank Adapters (LoRAs) for targeted domains such as music, weather, and sports. We introduce a novel one-pass decoding strategy that merges predictions from multiple LoRA adapters efficiently during the auto-regressive text generation process. Experimental results show significant improvements, reducing the Word Error Rate (WER) by 10% to 17% across all target domains compared to the original model, with minimal performance regression in out-of-domain settings (e.g., -1% on Librispeech test sets). We also demonstrate that DAS operates efficiently during inference, introducing an additional 9% increase in Real Time Factor (RTF) compared to the original model when inferring with three LoRA adapters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_12501 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Domain Adaptation Framework for Speech Recognition Systems with Only Synthetic data Tran, Minh Pang, Yutong Paul, Debjyoti Pandey, Laxmi Jiang, Kevin Guo, Jinxi Li, Ke Zhang, Shun Zhang, Xuedong Lei, Xin Audio and Speech Processing Sound We introduce DAS (Domain Adaptation with Synthetic data), a novel domain adaptation framework for pre-trained ASR model, designed to efficiently adapt to various language-defined domains without requiring any real data. In particular, DAS first prompts large language models (LLMs) to generate domain-specific texts before converting these texts to speech via text-to-speech technology. The synthetic data is used to fine-tune Whisper with Low-Rank Adapters (LoRAs) for targeted domains such as music, weather, and sports. We introduce a novel one-pass decoding strategy that merges predictions from multiple LoRA adapters efficiently during the auto-regressive text generation process. Experimental results show significant improvements, reducing the Word Error Rate (WER) by 10% to 17% across all target domains compared to the original model, with minimal performance regression in out-of-domain settings (e.g., -1% on Librispeech test sets). We also demonstrate that DAS operates efficiently during inference, introducing an additional 9% increase in Real Time Factor (RTF) compared to the original model when inferring with three LoRA adapters. |
| title | A Domain Adaptation Framework for Speech Recognition Systems with Only Synthetic data |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2501.12501 |