Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917580005965824 |
|---|---|
| author | Villatoro-Tello, Esaú Madikeri, Srikanth Zuluaga-Gomez, Juan Sharma, Bidisha Sarfjoo, Seyyed Saeed Nigmatulina, Iuliia Motlicek, Petr Ivanov, Alexei V. Ganapathiraju, Aravind |
| author_facet | Villatoro-Tello, Esaú Madikeri, Srikanth Zuluaga-Gomez, Juan Sharma, Bidisha Sarfjoo, Seyyed Saeed Nigmatulina, Iuliia Motlicek, Petr Ivanov, Alexei V. Ganapathiraju, Aravind |
| contents | In this paper, we perform an exhaustive evaluation of different representations to address the intent classification problem in a Spoken Language Understanding (SLU) setup. We benchmark three types of systems to perform the SLU intent detection task: 1) text-based, 2) lattice-based, and a novel 3) multimodal approach. Our work provides a comprehensive analysis of what could be the achievable performance of different state-of-the-art SLU systems under different circumstances, e.g., automatically- vs. manually-generated transcripts. We evaluate the systems on the publicly available SLURP spoken language resource corpus. Our results indicate that using richer forms of Automatic Speech Recognition (ASR) outputs, namely word-consensus-networks, allows the SLU system to improve in comparison to the 1-best setup (5.5% relative improvement). However, crossmodal approaches, i.e., learning from acoustic and text embeddings, obtains performance similar to the oracle setup, a relative improvement of 17.8% over the 1-best configuration, being a recommended alternative to overcome the limitations of working with automatically generated transcripts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_08489 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks Villatoro-Tello, Esaú Madikeri, Srikanth Zuluaga-Gomez, Juan Sharma, Bidisha Sarfjoo, Seyyed Saeed Nigmatulina, Iuliia Motlicek, Petr Ivanov, Alexei V. Ganapathiraju, Aravind Computation and Language Artificial Intelligence Sound Audio and Speech Processing I.2.7 In this paper, we perform an exhaustive evaluation of different representations to address the intent classification problem in a Spoken Language Understanding (SLU) setup. We benchmark three types of systems to perform the SLU intent detection task: 1) text-based, 2) lattice-based, and a novel 3) multimodal approach. Our work provides a comprehensive analysis of what could be the achievable performance of different state-of-the-art SLU systems under different circumstances, e.g., automatically- vs. manually-generated transcripts. We evaluate the systems on the publicly available SLURP spoken language resource corpus. Our results indicate that using richer forms of Automatic Speech Recognition (ASR) outputs, namely word-consensus-networks, allows the SLU system to improve in comparison to the 1-best setup (5.5% relative improvement). However, crossmodal approaches, i.e., learning from acoustic and text embeddings, obtains performance similar to the oracle setup, a relative improvement of 17.8% over the 1-best configuration, being a recommended alternative to overcome the limitations of working with automatically generated transcripts. |
| title | Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks |
| topic | Computation and Language Artificial Intelligence Sound Audio and Speech Processing I.2.7 |
| url | https://arxiv.org/abs/2212.08489 |