PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913387421630464 |
|---|---|
| author | Le, Trang Lazar, Daniel Kim, Suyoun Jiang, Shan Le, Duc Sagar, Adithya Livshits, Aleksandr Aly, Ahmed Shrivastava, Akshat |
| author_facet | Le, Trang Lazar, Daniel Kim, Suyoun Jiang, Shan Le, Duc Sagar, Adithya Livshits, Aleksandr Aly, Ahmed Shrivastava, Akshat |
| contents | Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explored end-to-end models to improve the quality and robustness of SLU models with Deliberation, however these models have remained autoregressive, resulting in higher latencies. In this work we introduce PRoDeliberation, a novel method leveraging a Connectionist Temporal Classification-based decoding strategy as well as a denoising objective to train robust non-autoregressive deliberation models. We show that PRoDeliberation achieves the latency reduction of parallel decoding (2-10x improvement over autoregressive models) while retaining the ability to correct Automatic Speech Recognition (ASR) mistranscriptions of autoregressive deliberation systems. We further show that the design of the denoising training allows PRoDeliberation to overcome the limitations of small ASR devices, and we provide analysis on the necessity of each component of the system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07823 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding Le, Trang Lazar, Daniel Kim, Suyoun Jiang, Shan Le, Duc Sagar, Adithya Livshits, Aleksandr Aly, Ahmed Shrivastava, Akshat Computation and Language Sound Audio and Speech Processing Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explored end-to-end models to improve the quality and robustness of SLU models with Deliberation, however these models have remained autoregressive, resulting in higher latencies. In this work we introduce PRoDeliberation, a novel method leveraging a Connectionist Temporal Classification-based decoding strategy as well as a denoising objective to train robust non-autoregressive deliberation models. We show that PRoDeliberation achieves the latency reduction of parallel decoding (2-10x improvement over autoregressive models) while retaining the ability to correct Automatic Speech Recognition (ASR) mistranscriptions of autoregressive deliberation systems. We further show that the design of the denoising training allows PRoDeliberation to overcome the limitations of small ASR devices, and we provide analysis on the necessity of each component of the system. |
| title | PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.07823 |