PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929237349367808 |
|---|---|
| author | Pandey, Rahul Ren, Roger Luo, Qi Liu, Jing Rastrow, Ariya Gandhe, Ankur Filimonov, Denis Strimel, Grant Stolcke, Andreas Bulyko, Ivan |
| author_facet | Pandey, Rahul Ren, Roger Luo, Qi Liu, Jing Rastrow, Ariya Gandhe, Ankur Filimonov, Denis Strimel, Grant Stolcke, Andreas Bulyko, Ivan |
| contents | End-to-End (E2E) automatic speech recognition (ASR) systems used in voice assistants often have difficulties recognizing infrequent words personalized to the user, such as names and places. Rare words often have non-trivial pronunciations, and in such cases, human knowledge in the form of a pronunciation lexicon can be useful. We propose a PROnunCiation-aware conTextual adaptER (PROCTER) that dynamically injects lexicon knowledge into an RNN-T model by adding a phonemic embedding along with a textual embedding. The experimental results show that the proposed PROCTER architecture outperforms the baseline RNN-T model by improving the word error rate (WER) by 44% and 57% when measured on personalized entities and personalized rare entities, respectively, while increasing the model size (number of trainable parameters) by only 1%. Furthermore, when evaluated in a zero-shot setting to recognize personalized device names, we observe 7% WER improvement with PROCTER, as compared to only 1% WER improvement with text-only contextual attention |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_17131 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers Pandey, Rahul Ren, Roger Luo, Qi Liu, Jing Rastrow, Ariya Gandhe, Ankur Filimonov, Denis Strimel, Grant Stolcke, Andreas Bulyko, Ivan Audio and Speech Processing Sound End-to-End (E2E) automatic speech recognition (ASR) systems used in voice assistants often have difficulties recognizing infrequent words personalized to the user, such as names and places. Rare words often have non-trivial pronunciations, and in such cases, human knowledge in the form of a pronunciation lexicon can be useful. We propose a PROnunCiation-aware conTextual adaptER (PROCTER) that dynamically injects lexicon knowledge into an RNN-T model by adding a phonemic embedding along with a textual embedding. The experimental results show that the proposed PROCTER architecture outperforms the baseline RNN-T model by improving the word error rate (WER) by 44% and 57% when measured on personalized entities and personalized rare entities, respectively, while increasing the model size (number of trainable parameters) by only 1%. Furthermore, when evaluated in a zero-shot setting to recognize personalized device names, we observe 7% WER improvement with PROCTER, as compared to only 1% WER improvement with text-only contextual attention |
| title | PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2303.17131 |