StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918474139303936 |
|---|---|
| author | Boito, Marcely Zanon Brun, Caroline Kim, Inyoung Proux, Denys Ait-Mokhtar, Salah Lagos, Nikolaos Meunier, Jean-Luc Calapodescu, Ioan |
| author_facet | Boito, Marcely Zanon Brun, Caroline Kim, Inyoung Proux, Denys Ait-Mokhtar, Salah Lagos, Nikolaos Meunier, Jean-Luc Calapodescu, Ioan |
| contents | LLMs and speech assistants are increasingly used for task-oriented interactions, yet their evaluation often relies on controlled scenarios that fail to capture the variability and complexity of real user requests. Drink ordering, for example, involves diverse named entities, drink types, sizes, customizations, and brand-specific terminology, as well as spontaneous speech phenomena such as hesitations and self-corrections. To address this gap, we introduce StarDrinks, a test set in English and Korean containing speech utterances features, transcriptions, and annotated slots. Our dataset supports speech-to-slots SLU, transcription-to-slots NLU, and speech-to-transcription ASR evaluation, providing a realistic benchmark for model robustness and generalization in a linguistically rich, real-world task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_26500 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario Boito, Marcely Zanon Brun, Caroline Kim, Inyoung Proux, Denys Ait-Mokhtar, Salah Lagos, Nikolaos Meunier, Jean-Luc Calapodescu, Ioan Computation and Language LLMs and speech assistants are increasingly used for task-oriented interactions, yet their evaluation often relies on controlled scenarios that fail to capture the variability and complexity of real user requests. Drink ordering, for example, involves diverse named entities, drink types, sizes, customizations, and brand-specific terminology, as well as spontaneous speech phenomena such as hesitations and self-corrections. To address this gap, we introduce StarDrinks, a test set in English and Korean containing speech utterances features, transcriptions, and annotated slots. Our dataset supports speech-to-slots SLU, transcription-to-slots NLU, and speech-to-transcription ASR evaluation, providing a realistic benchmark for model robustness and generalization in a linguistically rich, real-world task. |
| title | StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.26500 |