Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.04235 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914635228119040 |
|---|---|
| author | Li, Christopher Wang, Gary Kastner, Kyle Su, Heng Chen, Allen Rosenberg, Andrew Chen, Zhehuai Wu, Zelin Velikovich, Leonid Rondon, Pat Caseiro, Diamantino Aleksic, Petar |
| author_facet | Li, Christopher Wang, Gary Kastner, Kyle Su, Heng Chen, Allen Rosenberg, Andrew Chen, Zhehuai Wu, Zelin Velikovich, Leonid Rondon, Pat Caseiro, Diamantino Aleksic, Petar |
| contents | Automatic speech recognition (ASR) systems can suffer from poor recall for various reasons, such as noisy audio, lack of sufficient training data, etc.
Previous work has shown that recall can be improved by retrieving rewrite candidates from a large database of likely, contextually-relevant alternatives to the hypothesis text using nearest-neighbors search over embeddings of the ASR hypothesis text to correct and candidate corrections.
However, ASR-hypothesis-based retrieval can yield poor precision if the textual hypotheses are too phonetically dissimilar to the transcript truth. In this paper, we eliminate the hypothesis-audio mismatch problem by querying the correction database directly using embeddings derived from the utterance audio; the embeddings of the utterance audio and candidate corrections are produced by multimodal speech-text embedding networks trained to place the embedding of the audio of an utterance and the embedding of its corresponding textual transcript close together.
After locating an appropriate correction candidate using nearest-neighbor search, we score the candidate with its speech-text embedding distance before adding the candidate to the original n-best list.
We show a relative word error rate (WER) reduction of 6% on utterances whose transcripts appear in the candidate set, without increasing WER on general utterances. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_04235 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | High-precision Voice Search Query Correction via Retrievable Speech-text Embedings Li, Christopher Wang, Gary Kastner, Kyle Su, Heng Chen, Allen Rosenberg, Andrew Chen, Zhehuai Wu, Zelin Velikovich, Leonid Rondon, Pat Caseiro, Diamantino Aleksic, Petar Computation and Language Sound Audio and Speech Processing Automatic speech recognition (ASR) systems can suffer from poor recall for various reasons, such as noisy audio, lack of sufficient training data, etc. Previous work has shown that recall can be improved by retrieving rewrite candidates from a large database of likely, contextually-relevant alternatives to the hypothesis text using nearest-neighbors search over embeddings of the ASR hypothesis text to correct and candidate corrections. However, ASR-hypothesis-based retrieval can yield poor precision if the textual hypotheses are too phonetically dissimilar to the transcript truth. In this paper, we eliminate the hypothesis-audio mismatch problem by querying the correction database directly using embeddings derived from the utterance audio; the embeddings of the utterance audio and candidate corrections are produced by multimodal speech-text embedding networks trained to place the embedding of the audio of an utterance and the embedding of its corresponding textual transcript close together. After locating an appropriate correction candidate using nearest-neighbor search, we score the candidate with its speech-text embedding distance before adding the candidate to the original n-best list. We show a relative word error rate (WER) reduction of 6% on utterances whose transcripts appear in the candidate set, without increasing WER on general utterances. |
| title | High-precision Voice Search Query Correction via Retrievable Speech-text Embedings |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2401.04235 |