Large Language Model Should Understand Pinyin for Chinese ASR Error Correction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929507974250496 |
|---|---|
| author | Li, Yuang Qiao, Xiaosong Zhao, Xiaofeng Zhao, Huan Tang, Wei Zhang, Min Yang, Hao |
| author_facet | Li, Yuang Qiao, Xiaosong Zhao, Xiaofeng Zhao, Huan Tang, Wei Zhang, Min Yang, Hao |
| contents | Large language models can enhance automatic speech recognition systems through generative error correction. In this paper, we propose Pinyin-enhanced GEC, which leverages Pinyi, the phonetic representation of Mandarin Chinese, as supplementary information to improve Chinese ASR error correction. Our approach only utilizes synthetic errors for training and employs the one-best hypothesis during inference. Additionally, we introduce a multitask training approach involving conversion tasks between Pinyin and text to align their feature spaces. Experiments on the Aishell-1 and the Common Voice datasets demonstrate that our approach consistently outperforms GEC with text-only input. More importantly, we provide intuitive explanations for the effectiveness of PY-GEC and multitask training from two aspects: 1) increased attention weight on Pinyin features; and 2) aligned feature space between Pinyin and text hidden states. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13262 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Model Should Understand Pinyin for Chinese ASR Error Correction Li, Yuang Qiao, Xiaosong Zhao, Xiaofeng Zhao, Huan Tang, Wei Zhang, Min Yang, Hao Computation and Language Sound Audio and Speech Processing Large language models can enhance automatic speech recognition systems through generative error correction. In this paper, we propose Pinyin-enhanced GEC, which leverages Pinyi, the phonetic representation of Mandarin Chinese, as supplementary information to improve Chinese ASR error correction. Our approach only utilizes synthetic errors for training and employs the one-best hypothesis during inference. Additionally, we introduce a multitask training approach involving conversion tasks between Pinyin and text to align their feature spaces. Experiments on the Aishell-1 and the Common Voice datasets demonstrate that our approach consistently outperforms GEC with text-only input. More importantly, we provide intuitive explanations for the effectiveness of PY-GEC and multitask training from two aspects: 1) increased attention weight on Pinyin features; and 2) aligned feature space between Pinyin and text hidden states. |
| title | Large Language Model Should Understand Pinyin for Chinese ASR Error Correction |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2409.13262 |