An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917777181245440 |
|---|---|
| author | Tulsiani, Hitesh Chan, David M. Ghosh, Shalini Lalwani, Garima Pandey, Prabhat Bansal, Ankish Garimella, Sri Rastrow, Ariya Hoffmeister, Björn |
| author_facet | Tulsiani, Hitesh Chan, David M. Ghosh, Shalini Lalwani, Garima Pandey, Prabhat Bansal, Ankish Garimella, Sri Rastrow, Ariya Hoffmeister, Björn |
| contents | Dialog systems, such as voice assistants, are expected to engage with users in complex, evolving conversations. Unfortunately, traditional automatic speech recognition (ASR) systems deployed in such applications are usually trained to recognize each turn independently and lack the ability to adapt to the conversational context or incorporate user feedback. In this work, we introduce a general framework for ASR in dialog systems that can go beyond learning from single-turn utterances and learn over time how to adapt to both explicit supervision and implicit user feedback present in multi-turn conversations. We accomplish that by leveraging advances in student-teacher learning and context-aware dialog processing, and designing contrastive self-supervision approaches with Ohm, a new online hard-negative mining approach. We show that leveraging our new framework compared to traditional training leads to relative WER reductions of close to 10% in real-world dialog systems, and up to 26% on public synthetic data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10515 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems Tulsiani, Hitesh Chan, David M. Ghosh, Shalini Lalwani, Garima Pandey, Prabhat Bansal, Ankish Garimella, Sri Rastrow, Ariya Hoffmeister, Björn Audio and Speech Processing Artificial Intelligence Computation and Language Sound Dialog systems, such as voice assistants, are expected to engage with users in complex, evolving conversations. Unfortunately, traditional automatic speech recognition (ASR) systems deployed in such applications are usually trained to recognize each turn independently and lack the ability to adapt to the conversational context or incorporate user feedback. In this work, we introduce a general framework for ASR in dialog systems that can go beyond learning from single-turn utterances and learn over time how to adapt to both explicit supervision and implicit user feedback present in multi-turn conversations. We accomplish that by leveraging advances in student-teacher learning and context-aware dialog processing, and designing contrastive self-supervision approaches with Ohm, a new online hard-negative mining approach. We show that leveraging our new framework compared to traditional training leads to relative WER reductions of close to 10% in real-world dialog systems, and up to 26% on public synthetic data. |
| title | An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems |
| topic | Audio and Speech Processing Artificial Intelligence Computation and Language Sound |
| url | https://arxiv.org/abs/2409.10515 |