WASIL: In-the-Wild Arabic Spoken Interactions with LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909048550457344 |
|---|---|
| author | Ali, Zien Sheikh Mubarak, Hamdy Jung, Soon-Gyo Bhatti, Hunzalah Hassan Alam, Firoj Chowdhury, Shammur Absar |
| author_facet | Ali, Zien Sheikh Mubarak, Hamdy Jung, Soon-Gyo Bhatti, Hunzalah Hassan Alam, Firoj Chowdhury, Shammur Absar |
| contents | Large Language Models (LLMs) voice assistants are commonly built as cascaded Automatic Speech recognition (ASR) to LLM systems, where recognition errors can distort user intent. Dislikes may also arise from ambiguous, out-of-domain, or non-request turns, making it hard to isolate ASR effects. We release WASIL (it denotes connection or linking in Arabic): in-the-wild Arabic spoken interaction prompts with audio, ASR hypotheses, assistant responses, and explicit like/dislike feedback (8,529 turns; 14.2% dislikes), plus a 2,000-turn test set covering Modern Standard Arabic (MSA) and four major dialects with their labels. We provide low-cost gold transcripts via multi-ASR agreement-guided post-editing and annotate answerability (answerable, ambiguous/needs-clarification, unsupported, not-a-request/noise) to separate intrinsic unanswerability from ASR-induced degradation. Finally, we describe scalable reference-free evaluation of responses from ASR vs. gold transcripts using multi-judge LLM scoring. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_16364 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | WASIL: In-the-Wild Arabic Spoken Interactions with LLMs Ali, Zien Sheikh Mubarak, Hamdy Jung, Soon-Gyo Bhatti, Hunzalah Hassan Alam, Firoj Chowdhury, Shammur Absar Sound Artificial Intelligence Computation and Language 68T50 I.2.7 Large Language Models (LLMs) voice assistants are commonly built as cascaded Automatic Speech recognition (ASR) to LLM systems, where recognition errors can distort user intent. Dislikes may also arise from ambiguous, out-of-domain, or non-request turns, making it hard to isolate ASR effects. We release WASIL (it denotes connection or linking in Arabic): in-the-wild Arabic spoken interaction prompts with audio, ASR hypotheses, assistant responses, and explicit like/dislike feedback (8,529 turns; 14.2% dislikes), plus a 2,000-turn test set covering Modern Standard Arabic (MSA) and four major dialects with their labels. We provide low-cost gold transcripts via multi-ASR agreement-guided post-editing and annotate answerability (answerable, ambiguous/needs-clarification, unsupported, not-a-request/noise) to separate intrinsic unanswerability from ASR-induced degradation. Finally, we describe scalable reference-free evaluation of responses from ASR vs. gold transcripts using multi-judge LLM scoring. |
| title | WASIL: In-the-Wild Arabic Spoken Interactions with LLMs |
| topic | Sound Artificial Intelligence Computation and Language 68T50 I.2.7 |
| url | https://arxiv.org/abs/2605.16364 |