Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909453915258880 |
|---|---|
| author | Asano, Yuya Hassan, Sabit Sharma, Paras Sicilia, Anthony Atwell, Katherine Litman, Diane Alikhani, Malihe |
| author_facet | Asano, Yuya Hassan, Sabit Sharma, Paras Sicilia, Anthony Atwell, Katherine Litman, Diane Alikhani, Malihe |
| contents | General-purpose automatic speech recognition (ASR) systems do not always perform well in goal-oriented dialogue. Existing ASR correction methods rely on prior user data or named entities. We extend correction to tasks that have no prior user data and exhibit linguistic flexibility such as lexical and syntactic variations. We propose a novel context augmentation with a large language model and a ranking strategy that incorporates contextual information from the dialogue states of a goal-oriented conversational AI and its tasks. Our method ranks (1) n-best ASR hypotheses by their lexical and semantic similarity with context and (2) context by phonetic correspondence with ASR hypotheses. Evaluated in home improvement and cooking domains with real-world users, our method improves recall and F1 of correction by 34% and 16%, respectively, while maintaining precision and false positive rate. Users rated .8-1 point (out of 5) higher when our correction method worked properly, with no decrease due to false positives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06129 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI Asano, Yuya Hassan, Sabit Sharma, Paras Sicilia, Anthony Atwell, Katherine Litman, Diane Alikhani, Malihe Computation and Language Artificial Intelligence General-purpose automatic speech recognition (ASR) systems do not always perform well in goal-oriented dialogue. Existing ASR correction methods rely on prior user data or named entities. We extend correction to tasks that have no prior user data and exhibit linguistic flexibility such as lexical and syntactic variations. We propose a novel context augmentation with a large language model and a ranking strategy that incorporates contextual information from the dialogue states of a goal-oriented conversational AI and its tasks. Our method ranks (1) n-best ASR hypotheses by their lexical and semantic similarity with context and (2) context by phonetic correspondence with ASR hypotheses. Evaluated in home improvement and cooking domains with real-world users, our method improves recall and F1 of correction by 34% and 16%, respectively, while maintaining precision and false positive rate. Users rated .8-1 point (out of 5) higher when our correction method worked properly, with no decrease due to false positives. |
| title | Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2501.06129 |